GPU (Graphics Processing Unit) computing power dynamic adjustment method, device and equipment and storage medium

Abstract

The invention discloses a GPU computing power dynamic adjustment method and device, equipment and a storage medium, and relates to the technical field of GPU computing resource management.The operating state of GPU equipment in a GPU pool is obtained firstly, and then capacity expansion or capacity shrinkage is conducted on computing power resources of the GPU equipment in the GPU pool based on the operating state of the GPU equipment in the GPU pool and a preset multi-dimensional capacity expansion and shrinkage strategy. Therefore, by monitoring the running state of the GPU equipment and the resource use condition of the task in real time and combining a multi-dimensional capacity expansion and contraction strategy, the demand change of GPU resources can be accurately judged, and corresponding adjustment can be quickly made, so that the GPU resources can be dynamically allocated according to actual demands, and the effects of efficiently utilizing the GPU resources and improving the task execution efficiency are achieved. Therefore, waste of GPU resources can be avoided, operation cost can be reduced, flexibility and expandability of the system can be improved, and business requirements of different scales and complexity can be better met.

Description

Technical Field

[0001] This application relates to the field of GPU computing resource management technology, and in particular to a method, apparatus, device and storage medium for dynamically adjusting GPU computing power. Background Technology

[0002] In existing GPU pooling application scenarios, GPU computing power allocation mostly adopts a fixed configuration mode.

[0003] However, when the workload suddenly increases, the fixed GPU computing power cannot meet the task requirements, which will lead to a significant decrease in execution efficiency; while during periods of low workload, a large amount of GPU computing power is idle, resulting in resource waste and increased costs.

[0004] To address the aforementioned issues, existing methods for adjusting computing power often require recreating virtual machines or containers, and the adjustment process is time-consuming and unfriendly to business operations. Therefore, it is impossible to achieve real-time, dynamic elastic scaling up and down while maintaining business operations, making it difficult to adapt to rapid changes in business resource demands. Summary of the Invention

[0005] To address the above problems, this application provides a method for dynamically adjusting GPU computing power, including the following:

[0006] In a first aspect, this application provides a method for dynamically adjusting GPU computing power, the method comprising:

[0007] Get the running status of GPU devices within the GPU pool;

[0008] Based on the operating status of the GPU devices in the GPU pool and the preset multi-dimensional scaling strategy, the computing resources of the GPU devices in the GPU pool are expanded or reduced.

[0009] Optionally, obtaining the operating status of GPU devices within the GPU pool includes:

[0010] Get the GPU utilization, temperature, and GPU resource usage of the GPU devices in the GPU pool, as well as the current task's GPU resource usage.

[0011] Optionally, the preset multi-dimensional scaling strategy includes:

[0012] The allocation of computing resources is dynamically adjusted based on preset GPU utilization thresholds and the duration of reaching those thresholds, preset GPU temperature thresholds and the duration of reaching those thresholds, or the GPU resource usage of the current task.

[0013] Optionally, the step of scaling up or down the computing resources of the GPU devices in the GPU pool based on the operating status of the GPU devices in the GPU pool and a preset multi-dimensional scaling strategy includes:

[0014] The operating status of all GPU devices in the GPU pool is detected at preset intervals. If the detection results of m consecutive times indicate that the average GPU utilization of all GPU devices in the GPU pool is greater than or equal to the GPU utilization threshold, a computing power expansion operation is triggered. If the detection results of m consecutive times indicate that the average GPU utilization of all GPU devices in the GPU pool is less than the GPU utilization threshold, a computing power reduction operation is triggered.

[0015] The operating status of all GPU devices in the GPU pool is detected at preset intervals. If the detection results of n consecutive times indicate that the average GPU temperature of all GPU devices in the GPU pool is greater than or equal to the GPU temperature threshold, a computing power expansion operation is triggered. If the detection results of n consecutive times indicate that the average temperature of all GPU devices in the GPU pool is less than the GPU temperature threshold, a computing power reduction operation is triggered.

[0016] The GPU resource usage of the current task is detected at preset intervals. When the GPU resource usage of the current task is greater than or equal to the preset task GPU resource usage threshold for p consecutive detections, a computing power expansion operation is triggered. When the GPU resource usage of the current task is lower than the preset task GPU resource usage threshold for p consecutive detections, a computing power reduction operation is triggered.

[0017] Optionally, when performing computing power expansion operations, the GPU utilization rate can be adjusted or new replicas can be added based on the current computing power demand;

[0018] If no replica exists and the conditions for triggering a computing power expansion operation are met, the current GPU utilization rate is increased. When the computing power demand exceeds the preset replica expansion threshold, a new replica is added.

[0019] If a replica exists, and the computing power utilization rate of the current replica is greater than or equal to the preset replica expansion threshold within a preset time period, then a new replica will be added.

[0020] If there are multiple replicas, and the average computing power utilization rate of all current replicas is greater than or equal to the preset replica expansion threshold within a preset time period, then a new replica will be added.

[0021] When performing a computing power downsizing operation, adjust the GPU utilization or downsize the replicas according to the current computing power requirements;

[0022] If there are multiple replicas, and the average computing power utilization rate of all replicas is less than the preset replica scaling-down threshold within a preset time period, then the replicas will be scaled down.

[0023] If a replica exists, and the computing power utilization rate of the current replica is less than the preset replica scaling-down threshold within a preset time period, then the replica will be scaled down.

[0024] If no replica exists and the conditions for triggering compute power scaling down are met, the current GPU utilization is reduced. Optionally, the method further includes:

[0025] When resources in the GPU pool are insufficient, a backup GPU device is selected for initialization operations, including device startup, network configuration, component deployment, and resource registration. The backup GPU device is then added to the GPU pool to meet the expansion requirements.

[0026] Secondly, this application provides a GPU computing power dynamic adjustment device, the device comprising:

[0027] The acquisition unit is used to acquire the running status of GPU devices within the GPU pool.

[0028] The adjustment unit is used to expand or shrink the computing resources of the GPU devices in the GPU pool based on the operating status of the GPU devices in the GPU pool and the preset multi-dimensional expansion and contraction strategy.

[0029] Optionally, the acquisition unit is specifically used for:

[0030] Get the GPU utilization, temperature, and GPU resource usage of the GPU devices in the GPU pool, as well as the current task's GPU resource usage.

[0031] Optionally, the preset multi-dimensional scaling strategy includes:

[0032] The allocation of computing resources is dynamically adjusted based on preset GPU utilization thresholds and the duration of reaching those thresholds, preset GPU temperature thresholds and the duration of reaching those thresholds, or the GPU resource usage of the current task.

[0033] Optionally, the step of scaling up or down the computing resources of the GPU devices in the GPU pool based on the operating status of the GPU devices in the GPU pool and a preset multi-dimensional scaling strategy includes:

[0034] The operating status of all GPU devices in the GPU pool is detected at preset intervals. If the detection results of m consecutive times indicate that the average GPU utilization of all GPU devices in the GPU pool is greater than or equal to the GPU utilization threshold, a computing power expansion operation is triggered. If the detection results of m consecutive times indicate that the average GPU utilization of all GPU devices in the GPU pool is less than the GPU utilization threshold, a computing power reduction operation is triggered.

[0035] The operating status of all GPU devices in the GPU pool is detected at preset intervals. If the detection results of n consecutive times indicate that the average GPU temperature of all GPU devices in the GPU pool is greater than or equal to the GPU temperature threshold, a computing power expansion operation is triggered. If the detection results of n consecutive times indicate that the average temperature of all GPU devices in the GPU pool is less than the GPU temperature threshold, a computing power reduction operation is triggered.

[0036] The GPU resource usage of the current task is detected at preset intervals. When the GPU resource usage of the current task is greater than or equal to the preset task GPU resource usage threshold for p consecutive detections, a computing power expansion operation is triggered. When the GPU resource usage of the current task is lower than the preset task GPU resource usage threshold for p consecutive detections, a computing power reduction operation is triggered.

[0037] Optionally, when performing computing power expansion operations, the adjustment unit adjusts the GPU utilization rate or expands the capacity to include new replicas based on the current computing power requirements.

[0038] If no replica exists and the conditions for triggering a computing power expansion operation are met, the current GPU utilization rate is increased. When the computing power demand exceeds the preset replica expansion threshold, a new replica is added.

[0039] If a replica exists, and the computing power utilization rate of the current replica is greater than or equal to the preset replica expansion threshold within a preset time period, then a new replica will be added.

[0040] If there are multiple replicas, and the average computing power utilization rate of all current replicas is greater than or equal to the preset replica expansion threshold within a preset time period, then a new replica will be added.

[0041] When performing a computing power downsizing operation, adjust the GPU utilization or downsize the replicas according to the current computing power requirements;

[0042] If there are multiple replicas, and the average computing power utilization rate of all replicas is less than the preset replica scaling-down threshold within a preset time period, then the replicas will be scaled down.

[0043] If a replica exists, and the computing power utilization rate of the current replica is less than the preset replica scaling-down threshold within a preset time period, then the replica will be scaled down.

[0044] If no replica exists and the conditions for triggering computing power scaling down are met, the current GPU utilization rate is reduced. Optionally, the adjustment unit is further configured to select a backup GPU device for initialization operations when resources in the GPU pool are insufficient, including device startup, network configuration, component deployment, and resource registration, and add the backup GPU device to the GPU pool to meet the scaling up requirements.

[0045] Thirdly, this application provides an apparatus comprising a memory and a processor, the memory for storing instructions or code, and the processor for executing the instructions or code to cause the apparatus to perform the GPU computing power dynamic adjustment method described in any of the implementations of the first aspect.

[0046] Fourthly, this application provides a computer-readable storage medium storing code, wherein when the code is executed, a device running the code implements the GPU computing power dynamic adjustment method described in any of the implementations of the first aspect.

[0047] This application provides a method for dynamically adjusting GPU computing power. When executing the method, the operating status of GPU devices within a GPU pool is first obtained. Then, based on the operating status of the GPU devices in the GPU pool and a preset multi-dimensional scaling strategy, the computing power resources of the GPU devices in the GPU pool are scaled up or down. In this way, by monitoring the operating status of GPU devices and the resource usage of tasks in real time, combined with the multi-dimensional scaling strategy, changes in GPU resource demand can be accurately determined, and corresponding adjustments can be made quickly. This allows GPU resources to be dynamically allocated according to actual needs, achieving efficient utilization of GPU resources and improving task execution efficiency. This avoids wasting GPU resources, reduces operating costs, and enhances the flexibility and scalability of the system, better adapting to business needs of different scales and complexities. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in this embodiment or the prior art, the drawings used in the description of the embodiment or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 A flowchart illustrating a method for dynamically adjusting GPU computing power provided in this application embodiment;

[0050] Figure 2 This is a schematic diagram of a GPU computing power dynamic adjustment device provided in an embodiment of this application. Detailed Implementation

[0051] As described in the background section of this application, in GPU pooling applications, computing power allocation mostly adopts a fixed configuration mode. When the workload suddenly increases, the fixed GPU computing power cannot meet the demand, resulting in a significant decrease in execution efficiency; while during periods of low workload, a large amount of GPU computing power is idle, causing resource waste and increased costs. Existing computing power adjustment methods often require the re-creation of virtual machines / containers, and the adjustment process is time-consuming and not business-friendly. It is impossible to achieve real-time, dynamic, elastic scaling up and down while maintaining business operation, making it difficult to adapt to rapid changes in business needs.

[0052] For example, in deep learning training scenarios, some tasks have low GPU computing power requirements in the early stages of training, but due to the fixed static allocation of resources, a large amount of GPU resources are left idle and wasted; while other tasks have a surge in computing power requirements in the later stages of training, but cannot obtain more resources, thus affecting task execution efficiency. To solve the above technical problems, embodiments of this application provide a method for dynamically adjusting GPU computing power, the method comprising:

[0053] First, the operating status of the GPU devices within the GPU pool is acquired. Then, based on the operating status of the GPU devices in the GPU pool and a preset multi-dimensional scaling strategy, the computing resources of the GPU devices in the GPU pool are scaled up or down. In this way, by monitoring the operating status of GPU devices and the resource usage of tasks in real time, combined with the multi-dimensional scaling strategy, changes in GPU resource demand can be accurately determined, and corresponding adjustments can be made quickly. This allows GPU resources to be dynamically allocated according to actual needs, achieving efficient utilization of GPU resources and improving task execution efficiency. This avoids wasting GPU resources, reduces operating costs, and enhances the system's flexibility and scalability, better adapting to business needs of different scales and complexities.

[0054] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0055] Figure 1 This is a flowchart illustrating a method for dynamically adjusting GPU computing power as provided in an embodiment of this application. (In conjunction with...) Figure 1As shown, the GPU computing power dynamic adjustment method provided in this application embodiment may include:

[0056] S101. Obtain the running status of GPU devices in the GPU pool.

[0057] The operational status includes GPU utilization, temperature, and GPU resource usage of the GPU devices within the GPU pool. GPU utilization refers to the percentage of GPUs actually being used within a given time period, typically expressed as a percentage, such as 50% or 80%. Monitoring GPU utilization provides insight into the current load on the GPU devices. If GPU utilization remains consistently high, it indicates the devices are nearing full capacity and may require expansion; conversely, low GPU utilization may necessitate scaling down to free up resources.

[0058] GPU temperature refers to the actual temperature of a GPU device during operation, and it is one of the important indicators of the GPU's operating status. Excessively high or low temperatures can affect GPU performance and lifespan. By monitoring the temperature, potential heat dissipation problems or overheating risks can be detected early, and corresponding measures can be taken, such as increasing capacity to distribute the load or adjusting the cooling strategy.

[0059] The current GPU resource usage of a task refers to the proportion or amount of GPU resources actually consumed by the task. By monitoring the actual resource usage of a task, resource allocation can be dynamically adjusted to ensure that the task has sufficient resources to run efficiently, while avoiding resource waste.

[0060] S102. Based on the operating status of the GPU devices in the GPU pool and the preset multi-dimensional scaling strategy, the computing resources of the GPU devices in the GPU pool are expanded or reduced.

[0061] In this step, based on the operating status information of the GPU devices in the GPU pool obtained in step S101, and in conjunction with a preset multi-dimensional scaling strategy, the allocation of computing resources for each device in the GPU pool is dynamically adjusted. The preset multi-dimensional scaling strategy includes:

[0062] The allocation of computing resources is dynamically adjusted based on preset GPU utilization thresholds and the duration of reaching those thresholds, preset GPU temperature thresholds and the duration of reaching those thresholds, or the GPU resource usage of the current task.

[0063] The allocation of computing resources is dynamically adjusted based on a preset GPU utilization threshold and the duration of time the GPU utilization threshold is reached.

[0064] First, one or more GPU utilization thresholds are preset, such as 60% or 80%, and the duration for reaching these thresholds is set, for example, five consecutive probes with a five-minute interval between each probe. Then, the operating status of all GPU devices in the GPU pool is probed at preset intervals. When the results of m consecutive probes indicate that the average GPU utilization of all GPU devices in the GPU pool is greater than or equal to the GPU utilization threshold, a computing power expansion operation is triggered. For example, if the preset GPU utilization threshold is 80%, and the average utilization of all GPUs is higher than 80% for five consecutive probes (i.e., within 25 minutes), the system will allocate more GPU devices.

[0065] If m consecutive probes indicate that the average GPU utilization of all GPU devices in the GPU pool is less than the GPU utilization threshold, a computing power reduction operation is triggered. For example, if the preset GPU utilization threshold is 40%, and the average utilization of all GPUs is less than 40% within 5 consecutive probes (i.e., 25 minutes), the system will reclaim some idle GPU devices.

[0066] The above embodiments describe the implementation process of scaling up or down operations when there is only one GPU utilization threshold. When there are multiple preset GPU utilization thresholds, different detection windows and gradients need to be set for the scaling up or down strategies, for example:

[0067] Scaling strategy: If the average GPU utilization is above 60% after 6 consecutive probes every 5 minutes, then adjust the GPU utilization to 70%.

[0068] If the average GPU utilization is above 80% after six consecutive detections every 5 minutes, then adjust the GPU utilization to 100%.

[0069] If a probe is performed every 5 minutes and the average GPU utilization is above 90% for 6 consecutive probes, then a new copy is added.

[0070] Scaling down strategy: If the average GPU utilization is below 60% after 6 consecutive probes, then adjust the GPU utilization to 50%.

[0071] If the average GPU utilization is below 50% for 6 consecutive detections every 5 minutes, then adjust the GPU utilization to 40%.

[0072] If a probe is performed every 5 minutes, and the average GPU utilization is below 40% for 6 consecutive probes, then one copy is scaled down.

[0073] To dynamically adjust computing resource allocation based on preset GPU temperature thresholds and the duration for which these thresholds are reached, one or more GPU temperature thresholds are first set, such as 80°C or 90°C, and the duration for which these thresholds are reached is set, such as three consecutive probes with a 5-minute interval between each probe. If the average GPU temperature is higher than or equal to the preset temperature threshold after n consecutive probes, a computing power expansion operation is triggered. For example, if the preset GPU temperature threshold is 80°C, and the average GPU temperature of all GPUs is higher than 80°C within three consecutive probes (15 minutes), the system will add GPU devices to distribute the load and reduce the temperature.

[0074] If n consecutive probes show that the average GPU temperature of all GPUs is below a preset temperature threshold, a computing power reduction operation is triggered. For example, if the preset GPU temperature threshold is 60°C, and the average GPU temperature of all GPUs is below 60°C within 3 consecutive probes (i.e., 15 minutes), the system will reclaim some idle GPU devices.

[0075] The above embodiments describe the implementation process of scaling up or down operations when there is only one GPU temperature threshold. When there are multiple preset GPU temperature thresholds, different detection windows and gradients need to be set for the scaling up or down strategies, for example:

[0076] Scaling strategy: If the average GPU temperature of all GPUs is above 60°C after four consecutive detections every 5 minutes, the GPU computing power will be adjusted to 60%.

[0077] If the average GPU temperature of all GPUs is above 80°C after four consecutive tests every 5 minutes, the GPU computing power allocation will be adjusted to 70%.

[0078] If the average GPU temperature of all GPUs is above 90°C after four consecutive tests every 5 minutes, then immediately expand the capacity with a new GPU device or adjust the GPU computing power to 100%.

[0079] Reduced capacity strategy: If the average GPU temperature of all GPUs is below 60°C after four consecutive detections every 5 minutes, the GPU computing power allocation will be adjusted to 50%.

[0080] If the average GPU temperature of all GPUs is below 50°C after four consecutive tests every 5 minutes, then the GPU computing power allocation is adjusted to 40%.

[0081] If the average GPU temperature of all GPUs is below 40°C after four consecutive tests every 5 minutes, then one GPU device will be downsized or the GPU computing power will be adjusted to the minimum.

[0082] In one implementation of this application, in addition to dynamically adjusting the allocation of computing resources based on a preset GPU utilization threshold and the duration of reaching the GPU utilization threshold, a preset GPU temperature threshold and the duration of reaching the preset GPU temperature threshold, the allocation of computing resources can also be dynamically adjusted based on the GPU resource usage of the current task.

[0083] Specifically, the GPU resource usage of the current task is detected at preset intervals. When the GPU resource usage of the current task is greater than or equal to the preset task GPU resource usage threshold for p consecutive detections, a computing power expansion operation is triggered. For example, if the preset task resource usage threshold is 60%, and the task's GPU resource usage is higher than 60% for 4 consecutive detections (i.e., within 20 minutes), the system will increase the GPU computing power allocated to that task.

[0084] If p consecutive probes show that the GPU resource utilization of the current task is lower than the preset task resource utilization threshold, then the computing power of the current task will be reduced. For example, if the preset task resource utilization threshold is 40%, and the GPU resource utilization of the task is lower than 40% for 3 consecutive probes (i.e., within 15 minutes), then the system will reduce the GPU computing power allocated to that task.

[0085] Similar to the above process of dynamically adjusting computing resource allocation based on preset GPU utilization thresholds and the duration of reaching those thresholds, when dynamically adjusting computing resource allocation based on the current task's GPU resource usage, if there are multiple preset task GPU resource utilization thresholds, different detection windows and adjustment gradients need to be set for the scaling-up and scaling-down strategies. For example:

[0086] Scaling strategy: If the GPU resource utilization of a task is higher than 50% for four consecutive probes every 5 minutes, the GPU resource utilization of the task will be adjusted to 60%.

[0087] If the GPU resource utilization of a probe is higher than 70% for five consecutive probes every 5 minutes, then the GPU resource utilization of the probe will be adjusted to 80%.

[0088] If a probe is performed every 5 minutes and the GPU resource utilization rate is higher than 90% for 6 consecutive probe missions, then a new copy will be added.

[0089] Scaling down strategy: If the GPU resource utilization of a task is below 30% for three consecutive probes every 5 minutes, the GPU resource utilization of the task will be adjusted to 20%.

[0090] If the GPU resource utilization of a probe is below 20% for four consecutive probes every 5 minutes, then the GPU resource utilization of the probe will be adjusted to 10%.

[0091] If a probe is performed every 5 minutes, and the GPU resource utilization rate is below 10% for 5 consecutive probe missions, then one replica will be scaled down.

[0092] The above embodiments describe the process of dynamically adjusting computing resource allocation based on preset GPU utilization thresholds and the duration of reaching those thresholds, preset GPU temperature thresholds and the duration of reaching those thresholds, or the GPU resource usage of the current task. In one implementation of this application, the process of dynamically adjusting computing resource allocation can arbitrarily combine the above strategies to achieve dynamic adjustment of computing resource allocation. That is, dynamically adjusting computing resource allocation based on preset GPU utilization thresholds and the duration of reaching those thresholds, preset GPU temperature thresholds and the duration of reaching those thresholds, or dynamically adjusting computing resource allocation based on preset GPU temperature thresholds and the duration of reaching those thresholds, and the GPU resource usage of the current task, or dynamically adjusting computing resource allocation based on preset GPU utilization thresholds and the duration of reaching those thresholds, preset GPU temperature thresholds and the duration of reaching those thresholds, or the GPU resource usage of the current task.

[0093] In the case of the strategy combination, the conditions corresponding to all strategies in the combination need to be met simultaneously in order to adjust the allocation of computing resources.

[0094] For example, when dynamically adjusting the allocation of computing resources based on a preset GPU utilization threshold and the duration of reaching the GPU utilization threshold, a preset GPU temperature threshold and the duration of reaching the preset GPU temperature threshold, it is necessary to detect the operating status of all GPU devices in the GPU pool every preset time interval. When the detection results of m consecutive times indicate that the average GPU utilization of all GPU devices in the GPU pool is greater than or equal to the GPU utilization threshold, and when the detection results of n consecutive times indicate that the average GPU temperature of all GPU devices in the GPU pool is greater than or equal to the GPU temperature threshold, then the computing power expansion operation is triggered.

[0095] The operating status of all GPU devices in the GPU pool is detected at preset intervals. If the detection results of m consecutive times indicate that the average GPU utilization rate of all GPU devices in the GPU pool is less than the GPU utilization rate threshold, and if the detection results of n consecutive times indicate that the average GPU temperature of all GPU devices in the GPU pool is less than the GPU temperature threshold, then the computing power reduction operation is triggered.

[0096] For example, when dynamically adjusting the allocation of computing power resources based on a preset GPU temperature threshold, the duration of reaching the preset GPU temperature threshold, and the GPU resource usage of the current task, a computing power expansion operation is triggered when n consecutive detection results indicate that the average GPU temperature of all GPU devices in the GPU pool is greater than or equal to the GPU temperature threshold, and the GPU resource usage of the current task is greater than or equal to the preset task GPU resource usage threshold for p consecutive detection results.

[0097] When n consecutive detection results indicate that the average GPU temperature of all GPU devices in the GPU pool is less than the GPU temperature threshold and the GPU resource utilization of the current task is lower than the preset task GPU resource utilization threshold for p consecutive detection results, a computing power reduction operation is triggered.

[0098] For example, when dynamically adjusting computing power resource allocation based on a preset GPU utilization threshold and the duration of reaching the GPU utilization threshold, a preset GPU temperature threshold and the duration of reaching the preset GPU temperature threshold, or the GPU resource usage of the current task, it is necessary to detect the operating status of the GPU devices in the GPU pool every preset time interval. When the results of m consecutive detections indicate that the average GPU utilization of all GPU devices in the GPU pool is greater than or equal to the GPU utilization threshold, and the results of n consecutive detections indicate that the GPU temperature of the GPU devices in the GPU pool is greater than or equal to the GPU temperature threshold, and when the GPU resource utilization of the current task is greater than or equal to the preset task GPU resource utilization threshold for p consecutive detections, then a computing power expansion operation is triggered.

[0099] The operating status of the GPU devices in the GPU pool is detected at preset intervals. When the detection results of m consecutive times indicate that the average GPU utilization rate of all GPU devices in the GPU pool is less than the GPU utilization rate threshold, the detection results of n consecutive times indicate that the temperature of the GPU devices in the GPU pool is less than the GPU temperature threshold, and the GPU resource utilization rate of the current task is lower than the preset task GPU resource utilization rate threshold for p consecutive detection results, a computing power reduction operation is triggered.

[0100] In one implementation of this application, when performing a computing power expansion operation, the GPU utilization rate is adjusted or a new replica is added based on the computing power demand of the current replica. If no replica exists and the conditions for triggering the computing power expansion operation are met, the current GPU utilization rate is increased. When the computing power demand exceeds a preset replica expansion threshold, a new replica is added. If one replica exists and the computing power utilization rate of the current replica is greater than or equal to the preset replica expansion threshold within a preset time period, a new replica is added. If multiple replicas exist and the average computing power utilization rate of all current replicas is greater than or equal to the preset replica expansion threshold within a preset time period, a new replica is added. For example, when no replica exists and the conditions for triggering the computing power expansion operation are met, the operating status of all GPU devices in the GPU pool is checked every preset time interval. The detection process is as follows: If m consecutive detections indicate that the average GPU utilization of all GPU devices in the GPU pool is greater than or equal to the GPU utilization threshold, or if the operating status of all GPU devices in the GPU pool is detected at preset time intervals, and n consecutive detections indicate that the average GPU temperature of all GPU devices in the GPU pool is greater than or equal to the GPU temperature threshold, or if the GPU resource usage of the current task is detected at preset time intervals, and the GPU resource utilization of the current task is greater than or equal to the preset task GPU resource utilization threshold for p consecutive detections, or if any combination of the above conditions is met, or if all of the above conditions are met, then the current GPU utilization is increased. If the computing power demand exceeds the preset replica expansion threshold, then a new replica is added. For example, if there are no replicas, the preset GPU utilization threshold is 60%, and the average utilization of all GPUs is higher than 60% for 5 consecutive detections (i.e., within 25 minutes), then the current GPU utilization is increased to 80%; however, if the average utilization of all GPUs is greater than the preset replica expansion threshold (e.g., the preset replica expansion threshold is 80%) for 25 consecutive detections, then a new replica is added. If a replica exists, and the computing power utilization rate of the current replica is greater than or equal to the preset replica expansion threshold within a preset time period, then a new replica is added. If multiple replicas exist, and the average computing power utilization rate of all current replicas is greater than or equal to the preset replica expansion threshold within a preset time period, then a new replica is added.

[0101] When performing a computing power reduction operation, the GPU utilization rate is adjusted or the replica is reduced based on the computing power requirements of the current replica. If multiple replicas exist, and the average computing power utilization rate of all current replicas is less than a preset replica reduction threshold within a preset time period, then the replica is reduced. If one replica exists, and the computing power utilization rate of the current replica is less than the preset replica reduction threshold within a preset time period, then the replica is reduced. If no replica exists, and the conditions for triggering computing power reduction are met, such as when m consecutive detection results indicate that the average GPU utilization rate of all GPU devices in the GPU pool is less than the GPU utilization rate threshold, or n consecutive detection results indicate that the average temperature of all GPU devices in the GPU pool is less than the GPU temperature threshold, or when the GPU resource utilization rate of the current task is lower than the preset task GPU resource utilization rate threshold for p consecutive detection results, or any combination of two of the above conditions is met, or all of the above conditions are met, then the current GPU utilization rate is reduced.

[0102] Thus, when task load increases, adjusting GPU utilization or adding new replicas ensures that GPU resources can meet task demands, preventing decreased task execution efficiency due to insufficient resources. When task load decreases, adjusting GPU utilization or downsizing replicas releases excess GPU resources, avoiding resource waste and significantly improving overall GPU resource utilization. Furthermore, dynamically adjusting GPU utilization and replica count avoids resource idleness issues that may occur under fixed resource allocation models, reducing unnecessary energy consumption and equipment maintenance costs. During periods of low task load, timely downsizing replicas frees up excess GPU resources for other tasks or backup resource pools, further optimizing resource allocation and reducing operating costs.

[0103] In one implementation of this application, when the GPU resources in the resource pool are insufficient to meet the computing power requirements of the current task, the system automatically selects a backup GPU device from the backup devices and initializes it to quickly expand the resource capacity of the GPU pool. Specific operations include device startup, network configuration, component deployment, and resource registration, ensuring that the backup GPU device can be successfully added to the GPU pool and put into use, thereby effectively meeting the task's expansion requirements.

[0104] This mechanism ensures that the system can quickly respond and replenish the required GPU resources when resources in the resource pool are insufficient. By selecting GPU devices from the backup devices and performing initialization operations, the system can quickly expand the resource capacity of the GPU pool without affecting the operation of existing tasks. This process not only improves the flexibility and scalability of the system, but also ensures that tasks can obtain sufficient computing power support during peak resource demand periods, thereby guaranteeing efficient task execution and stable system operation. The above are some specific implementations of a dynamic GPU computing power adjustment method provided in the embodiments of this application. Based on this, this application also provides a corresponding device. The device provided in the embodiments of this application will be described below from the perspective of functional modularity.

[0105] Figure 2 This is a schematic diagram of a GPU computing power dynamic adjustment device provided in an embodiment of this application. (Combined with...) Figure 2 As shown in the embodiment of this application, the GPU computing power dynamic adjustment device 200 includes:

[0106] Acquisition unit 210 is used to acquire the running status of GPU devices in the GPU pool;

[0107] The adjustment unit 220 is used to expand or shrink the computing resources of the GPU devices in the GPU pool based on the operating status of the GPU devices in the GPU pool and the preset multi-dimensional expansion and contraction strategy.

[0108] In one implementation of this application embodiment, the acquisition unit is specifically used for:

[0109] Get the GPU utilization, temperature, and GPU resource usage of the GPU devices in the GPU pool, as well as the current task's GPU resource usage.

[0110] In one implementation of this application, the preset multi-dimensional scaling strategy includes:

[0111] The allocation of computing resources is dynamically adjusted based on preset GPU utilization thresholds and the duration of reaching those thresholds, preset GPU temperature thresholds and the duration of reaching those thresholds, or the GPU resource usage of the current task.

[0112] In one implementation of this application, the step of expanding or shrinking the computing resources of the GPU devices in the GPU pool based on the operating status of the GPU devices in the GPU pool and a preset multi-dimensional scaling strategy includes:

[0113] The operating status of the GPU devices in the GPU pool is detected at preset intervals. If the detection results for m consecutive times indicate that the GPU utilization rate of the GPU devices in the GPU pool is greater than or equal to the GPU utilization rate threshold, a computing power expansion operation is triggered. If the detection results for m consecutive times indicate that the GPU utilization rate of the GPU devices in the GPU pool is less than the GPU utilization rate threshold, a computing power reduction operation is triggered.

[0114] The operating status of the GPU devices in the GPU pool is detected at preset intervals. If the detection results show that the GPU temperature of the GPU devices in the GPU pool is greater than or equal to the GPU temperature threshold, the computing power expansion operation is triggered. If the detection results show that the GPU temperature of the GPU devices in the GPU pool is less than the GPU temperature threshold, the computing power reduction operation is triggered.

[0115] Every preset time interval, the GPU resource usage of the current task is detected. When the GPU resource usage of the current task is greater than or equal to the preset task GPU resource usage threshold for p consecutive detections, a computing power expansion operation is triggered. When the GPU resource usage of the current task is lower than the preset task GPU resource usage threshold for p consecutive detections, a computing power reduction operation is triggered.

[0116] In one implementation of this application embodiment, when the adjustment unit performs a computing power expansion operation, it adjusts the GPU utilization rate or expands the capacity to include new copies based on the current computing power requirements.

[0117] If no replica exists and the conditions for triggering a computing power expansion operation are met, the current GPU utilization rate is increased. When the computing power demand exceeds the preset replica expansion threshold, a new replica is added.

[0118] If a replica exists, and the computing power utilization rate of the current replica is greater than or equal to the preset replica expansion threshold within a preset time period, then a new replica will be added.

[0119] If there are multiple replicas, and the average computing power utilization rate of all current replicas is greater than or equal to the preset replica expansion threshold within a preset time period, then a new replica will be added.

[0120] When performing a computing power downsizing operation, adjust the GPU utilization or downsize the replicas according to the current computing power requirements;

[0121] If there are multiple replicas, and the average computing power utilization rate of all replicas is less than the preset replica scaling-down threshold within a preset time period, then the replicas will be scaled down.

[0122] If a replica exists, and the computing power utilization rate of the current replica is less than the preset replica scaling-down threshold within a preset time period, then the replica will be scaled down.

[0123] If no replica exists and the conditions for triggering computing power scaling down are met, the current GPU utilization rate is reduced. In one implementation of this application embodiment, the adjustment unit is further configured to, when resources in the GPU pool are insufficient, select a backup GPU device for initialization operations, including device startup, network configuration, component deployment, and resource registration, and add the backup GPU device to the GPU pool to meet the scaling requirements.

[0124] This application also provides corresponding devices and computer storage media for implementing the solutions provided in this application.

[0125] The device includes a memory and a processor. The memory stores instructions or code, and the processor executes the instructions or code to cause the device to perform the method described in any embodiment of this application.

[0126] The computer storage medium stores code, and when the code is run, the device running the code implements the method described in any embodiment of this application.

[0127] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0128] It is understood that in the specific embodiments of this application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved need to obtain user permission or consent when the above embodiments of this application are applied to specific products or technologies, and the collection, use and processing of related data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0129] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0130] It should also be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the device and apparatus embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The device and apparatus embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components indicated as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0131] The above description is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for dynamically adjusting GPU computing power, characterized in that, The method includes: Get the running status of GPU devices within the GPU pool; Based on the operating status of the GPU devices in the GPU pool and the preset multi-dimensional scaling strategy, the computing resources of the GPU devices in the GPU pool are expanded or reduced.

2. The method according to claim 1, characterized in that, The process of obtaining the operating status of GPU devices within the GPU pool includes: Get the GPU utilization, temperature, and GPU resource usage of the GPU devices in the GPU pool, as well as the current task's GPU resource usage.

3. The method according to claim 1, characterized in that, The preset multi-dimensional scaling strategy includes: The allocation of computing resources is dynamically adjusted based on preset GPU utilization thresholds and the duration of reaching those thresholds, preset GPU temperature thresholds and the duration of reaching those thresholds, or the GPU resource usage of the current task.

4. The method according to claim 3, characterized in that, The process of expanding or shrinking the computing resources of GPU devices in the GPU pool based on the operating status of GPU devices in the GPU pool and a preset multi-dimensional scaling strategy includes: The operating status of all GPU devices in the GPU pool is detected at preset intervals. If the detection results of m consecutive times indicate that the average GPU utilization of all GPU devices in the GPU pool is greater than or equal to the GPU utilization threshold, a computing power expansion operation is triggered. If the detection results of m consecutive times indicate that the average GPU utilization of all GPU devices in the GPU pool is less than the GPU utilization threshold, a computing power reduction operation is triggered. The operating status of all GPU devices in the GPU pool is detected at preset intervals. If the detection results of n consecutive times indicate that the average GPU temperature of all GPU devices in the GPU pool is greater than or equal to the GPU temperature threshold, a computing power expansion operation is triggered. If the detection results of n consecutive times indicate that the average temperature of all GPU devices in the GPU pool is less than the GPU temperature threshold, a computing power reduction operation is triggered. The GPU resource usage of the current task is detected at preset intervals. When the GPU resource usage of the current task is greater than or equal to the preset task GPU resource usage threshold for p consecutive detections, a computing power expansion operation is triggered. When the GPU resource usage of the current task is lower than the preset task GPU resource usage threshold for p consecutive detections, a computing power reduction operation is triggered.

5. The method according to claim 4, characterized in that, When performing computing power expansion operations, adjust the GPU utilization rate or add new replicas according to the current computing power demand; If no replica exists and the conditions for triggering a computing power expansion operation are met, the current GPU utilization rate is increased. When the computing power demand exceeds the preset replica expansion threshold, a new replica is added. If a replica exists, and the computing power utilization rate of the current replica is greater than or equal to the preset replica expansion threshold within a preset time period, then a new replica will be added. If there are multiple replicas, and the average computing power utilization rate of all current replicas is greater than or equal to the preset replica expansion threshold within a preset time period, then a new replica will be added. When performing a computing power downsizing operation, adjust the GPU utilization or downsize the replicas according to the current computing power requirements; If there are multiple replicas, and the average computing power utilization rate of all replicas is less than the preset replica scaling-down threshold within a preset time period, then the replicas will be scaled down. If a replica exists, and the computing power utilization rate of the current replica is less than the preset replica scaling-down threshold within a preset time period, then the replica will be scaled down. If no replica exists and the conditions for triggering compute power scaling down are met, reduce the current GPU utilization.

6. The method according to claim 1, characterized in that, The method further includes: When resources in the GPU pool are insufficient, a backup GPU device is selected for initialization operations, including device startup, network configuration, component deployment, and resource registration. The backup GPU device is then added to the GPU pool to meet the expansion requirements.

7. A device for dynamically adjusting GPU computing power, characterized in that, The device includes: The acquisition unit is used to acquire the running status of GPU devices within the GPU pool. The adjustment unit is used to expand or shrink the computing resources of the GPU devices in the GPU pool based on the operating status of the GPU devices in the GPU pool and the preset multi-dimensional expansion and contraction strategy.

8. The apparatus according to claim 7, characterized in that, The acquisition unit is specifically used for: Get the GPU utilization, temperature, and GPU resource usage of the GPU devices in the GPU pool, as well as the current task's GPU resource usage.

9. A computing device, characterized in that, The computing device includes: a memory and a processor; The memory is used to store computer programs; The processor is configured to implement the method as described in any one of claims 1 to 6 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 6.

Images