GPU power consumption regulation and control method and device, computer equipment and storage medium

Abstract

The invention relates to a GPU power consumption regulation and control method and device, computer equipment and a storage medium. Comprising the following steps: acquiring parameter information and GPU resource information of a target model, and calculating effective computing power required to be provided by each GPU according to the parameter information and the GPU resource information; calculating the target core frequency of the GPU according to the effective computing power required to be provided by the GPU; periodically acquiring current state information of the GPU, and judging whether the current state information exceeds a preset state threshold value or not; and in response to the fact that the current state information exceeds a preset state threshold value and the difference value between the current core frequency of the GPU and the target core frequency meets a preset range, determining the target core voltage of the GPU based on the target core frequency, and adjusting the working voltage of the GPU according to the target core frequency and the target core voltage. By adopting the method, the core voltage of the GPU can be dynamically adjusted so as to improve the heat dissipation effect and improve the utilization rate of the GPU.

Description

Technical Field

[0001] The present application relates to the field of server technology, and in particular to a GPU power consumption control method, device, computer equipment and storage medium. Background Art

[0002] In the field of AIGC (Artificial Intelligence Generated Content), especially in the application of large-scale language models, a large number of floating-point operations and memory access are required, and high-performance computing hardware such as GPU (Graphics Processing Unit) has become a key technical support. With its powerful parallel processing capabilities, GPU can complete a large number of complex matrix operations and deep learning tasks in a short time, greatly accelerating the model training and reasoning process.

[0003] However, high-performance GPUs usually consume a lot of power. For example, the thermal design power (TDP) of the NVIDIA H800 series can reach 700W. This means that the total power consumption of a single server or cabinet has increased significantly, which in turn places higher demands on the overall energy consumption of the data center. In addition, high power consumption also generates a lot of heat, requiring expensive and complex cooling systems to maintain the normal operation of the equipment. In actual applications, when the GPU is running at full load, due to the heat dissipation bottleneck, the core frequency may be automatically reduced to avoid overheating, thereby affecting computing performance. These problems are particularly evident in densely deployed data centers, where space limitations and heat dissipation efficiency become key factors restricting the performance of GPUs.

[0004] Therefore, how to control energy consumption and improve heat dissipation while ensuring high performance has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] Based on this, it is necessary to provide a GPU power consumption control method, device, computer equipment and storage medium that can dynamically adjust the GPU core voltage to address the above technical problems.

[0006] On the one hand, a method for controlling GPU power consumption is provided, the method comprising:

[0007] Obtain parameter information and GPU resource information of the target model, and calculate the effective computing power that each GPU needs to provide based on the parameter information and GPU resource information;

[0008] Calculating the target core frequency of the GPU according to the effective computing power that the GPU needs to provide;

[0009] Periodically obtaining current status information of the GPU, and determining whether the current status information exceeds a preset status threshold;

[0010] In response to the current state information exceeding a preset state threshold, and the difference between the current core frequency of the GPU and the target core frequency satisfies a preset range, the target core voltage of the GPU is determined based on the target core frequency, and the operating voltage of the GPU is adjusted according to the target core frequency and the target core voltage.

[0011] In one embodiment, before obtaining the parameter information and GPU resource information of the target model, the process further includes:

[0012] Determine application scenarios based on business needs, including model training, model reasoning, high-performance computing, and image processing;

[0013] According to the application scenario, GPU resources are scheduled based on the disaster recovery strategy.

[0014] In one embodiment, the parameter information of the target model includes the model parameter quantity and the data set size, and the obtaining the parameter information and GPU resource information of the target model, and calculating the effective computing power that the GPU needs to provide according to the parameter information and the GPU resource information, includes:

[0015] Calculate the number of floating-point operations required for training according to the model parameters and the size of the data set;

[0016] Calculate the computing power required for training according to the number of floating-point operations required for the training and the preset training time;

[0017] Based on the computing power required for the training, the effective computing power that each GPU needs to provide is calculated according to the GPU resource information.

[0018] In one embodiment, the GPU resource information includes the GPU model and quantity; the computing power required for the training and calculating the effective computing power required to be provided by each GPU according to the GPU resource information include:

[0019] Determine whether the GPUs in the current server are of the same model;

[0020] If yes, then calculate the effective computing power that each GPU needs to provide based on the computing power required for the training and the number of GPUs;

[0021] If not, the effective computing power that each GPU needs to provide is calculated based on the computing power required for the training and the computing power weights corresponding to the pre-configured GPU models.

[0022] In one embodiment, calculating the target core frequency of the GPU according to the effective computing power that the GPU needs to provide includes:

[0023] The target core frequency of the GPU is calculated using the peak computing power calculation formula:

[0024] Peak FLOPS = F clk *N SM *T ins *2

[0025] Where PeakFLOPS is the effective computing power that the GPU needs to provide, F clk is the target core frequency of the GPU, N SM is the number of computing units of the GPU, T ins The instruction throughput for a specific data type.

[0026] In one embodiment, periodically obtaining the current state information of the GPU and determining whether the current state information exceeds a preset state threshold includes:

[0027] Obtaining a core voltage value range of the GPU, and calculating a core frequency dynamic change rate of the GPU according to a difference between a maximum value and a minimum value of the core voltage value range;

[0028] Obtaining the maximum core frequency of the GPU, and calculating the core frequency value range of the GPU according to the core frequency dynamic change rate and the maximum core frequency;

[0029] Periodically acquiring a current temperature of the GPU, and comparing the current temperature with a preset first temperature threshold and a second temperature threshold;

[0030] In response to the current temperature being less than the first temperature threshold, taking a maximum value of a core voltage value range of the GPU as a target core voltage of the GPU;

[0031] In response to the current temperature being greater than the second temperature threshold, taking a minimum value of a core voltage value range of the GPU as a target core voltage of the GPU;

[0032] In response to the current temperature being less than or equal to the second temperature threshold and greater than or equal to the first temperature threshold, the current core frequency of the GPU is obtained; if the current core frequency is within the core frequency value range of the GPU, it is determined that the difference between the current core frequency of the GPU and the target core frequency satisfies a preset range.

[0033] In one embodiment, determining a target core voltage of a GPU based on the target core frequency, and adjusting an operating voltage of the GPU according to the target core frequency and the target core voltage, comprises:

[0034] Determining a target core voltage corresponding to the target core frequency based on a correlation between a core frequency and a core voltage of the GPU;

[0035] The operating voltage of the GPU is set according to the target core voltage through the SMBus bus.

[0036] On the other hand, a GPU power consumption control device is provided, the device comprising:

[0037] A first computing module is used to obtain parameter information and GPU resource information of the target model, and calculate the effective computing power that each GPU needs to provide according to the parameter information and GPU resource information;

[0038] A second calculation module, used for calculating the target core frequency of the GPU according to the effective computing power that the GPU needs to provide;

[0039] A determination module, used for periodically acquiring current status information of the GPU and determining whether the current status information exceeds a preset status threshold;

[0040] A setting module is used to determine a target core voltage of the GPU based on the target core frequency in response to the current state information exceeding a preset state threshold and the difference between the current core frequency of the GPU and the target core frequency satisfies a preset range, and adjust the operating voltage of the GPU according to the target core frequency and the target core voltage.

[0041] In another aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0042] Obtain parameter information and GPU resource information of the target model, and calculate the effective computing power that each GPU needs to provide based on the parameter information and GPU resource information;

[0043] Calculating the target core frequency of the GPU according to the effective computing power that the GPU needs to provide;

[0044] Periodically obtaining current status information of the GPU, and determining whether the current status information exceeds a preset status threshold;

[0045] In response to the current state information exceeding a preset state threshold, and the difference between the current core frequency of the GPU and the target core frequency satisfies a preset range, the target core voltage of the GPU is determined based on the target core frequency, and the operating voltage of the GPU is adjusted according to the target core frequency and the target core voltage.

[0046] In another aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0047] Obtain parameter information and GPU resource information of the target model, and calculate the effective computing power that each GPU needs to provide based on the parameter information and GPU resource information;

[0048] Calculating the target core frequency of the GPU according to the effective computing power that the GPU needs to provide;

[0049] Periodically obtaining current status information of the GPU, and determining whether the current status information exceeds a preset status threshold;

[0050] In response to the current state information exceeding a preset state threshold, and the difference between the current core frequency of the GPU and the target core frequency satisfies a preset range, the target core voltage of the GPU is determined based on the target core frequency, and the operating voltage of the GPU is adjusted according to the target core frequency and the target core voltage.

[0051] The above-mentioned GPU power consumption control method, device, computer equipment and storage medium calculate the effective computing power that each GPU needs to provide by obtaining parameter information of the target model and GPU resource information, further determine the target core frequency and corresponding target core voltage of the GPU, and dynamically adjust the operating frequency and operating voltage of the GPU according to the current status information of the GPU, so as to better adapt to the changes in computing power requirements in different application scenarios, improve GPU utilization, improve the heat dissipation effect of the whole machine, and extend the life of the GPU card. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A schematic diagram of a process of a GPU power consumption control method in one embodiment;

[0053] Figure 2 A schematic diagram of a process of a GPU power consumption control method according to another embodiment;

[0054] Figure 3 A schematic diagram of a process of a GPU power consumption control method according to another embodiment;

[0055] Figure 4 A schematic diagram of the internal structure of a GPU in one embodiment;

[0056] Figure 5 A structural block diagram of a GPU power consumption control device in one embodiment;

[0057] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] AIGC is an application model of artificial intelligence technology based on deep neural network models. It uses deep learning algorithms to perform a large number of calculations such as matrix multiplication and convolution. These calculations require a large number of floating-point operations and memory access. The GPU architecture is designed to accelerate these calculations. Each core of the GPU can perform multiple floating-point operations, which allows the GPU to complete a large number of calculations in a short period of time, but also leads to high power consumption of the GPU. In addition, the GPU generates a lot of heat during the training / inference process, which requires more cooling equipment to maintain a stable operating temperature. The cooling equipment also requires electricity. Therefore, the high power consumption and heat dissipation of GPU training large models will increase the electricity cost of the GPU.

[0060] The ability to increase the GPU frequency is mainly limited by TDP power consumption, that is, the core frequency gradually decreases as the GPU temperature increases. Therefore, increasing the GPU frequency requires a high heat dissipation capacity of the entire system. When the GPU load increases, if the heat dissipation performance of the entire system is poor, the GPU will not increase the core frequency, and thus cannot bring out the true performance of the graphics card.

[0061] In one embodiment, Figure 1 As shown, a GPU power consumption control method is provided, comprising the following steps:

[0062] Step S1, obtaining parameter information and GPU resource information of the target model, and calculating the effective computing power that each GPU needs to provide based on the parameter information and GPU resource information.

[0063] In one possible implementation, before obtaining the parameter information and GPU resource information of the target model, it also includes: determining application scenarios according to business needs, the application scenarios including model training, model reasoning, high-performance computing and image processing; according to the application scenarios, scheduling GPU resources based on disaster recovery strategies.

[0064] Based on the above steps, GPU resources can be allocated according to specific application scenarios to ensure full utilization of resources while meeting performance requirements, which helps to improve the efficiency and energy efficiency of the data center.

[0065] In one possible implementation mode, the specific steps of obtaining parameter information and GPU resource information of the target model, and calculating the effective computing power that the GPU needs to provide based on the parameter information and GPU resource information include: calculating the number of floating-point operations required for training based on the model parameter quantity and the size of the data set; calculating the computing power required for training based on the number of floating-point operations required for training and the preset training time; based on the computing power required for training, calculating the effective computing power that each GPU needs to provide based on the GPU resource information.

[0066] For example, taking a large text generation AI model as an example, its model to be trained has 280 billion parameters, and the number of training sample tokens reaches 13 trillion. Assuming that each calculation requires 6 floating-point operations for each token and each parameter, the total number of floating-point operations required for training is:

[0067] The number of floating point operations required for training = number of parameters * number of sample tokens * calculation depth

[0068] =2800×10 9 ×130000×10 9 ×6=2184×10 22 FLOPs;

[0069] Based on the calculated number of floating-point operations and the preset training time (for example, 30 days, or 2592000 seconds), the computing power required for training can be calculated:

[0070] Computing power required for training = number of floating point operations required for training / training time

[0071] =2184×10 22 FLOPs / 2592000=8425926TFLOPS;

[0072] Based on the calculated computing power required for training, GPU resource information can be obtained based on the scheduled GPU resources, thereby calculating the effective computing power that each GPU needs to provide.

[0073] Based on the above steps, the computing power required for training is calculated according to the customer's actual training / inference model parameters and data scale, and the effective computing power required for each GPU is calculated according to the GPU resource information, thereby achieving effective allocation of GPU resources, ensuring that each GPU can operate in its optimal state, improving GPU utilization, and enhancing energy conversion efficiency, and achieving green and low-carbon development of data centers.

[0074] In one possible implementation, the effective computing power that each GPU needs to provide is calculated based on the computing power required for training and the GPU resource information, including: determining whether the GPUs in the current server are of the same model; if so, calculating the effective computing power that each GPU needs to provide based on the computing power required for training and the number of GPUs; if not, calculating the effective computing power that each GPU needs to provide based on the computing power required for training and the computing power weights corresponding to each pre-configured GPU model.

[0075] Specifically, if according to the results of GPU resource allocation, the GPUs in the current server are all of the same model, then the effective computing power that each GPU needs to provide is calculated based on the computing power required for training and the number of GPUs. For example, a data center is equipped with 3,500 4U8-card AI servers equipped with A100 GPUs. The effective computing power required for each A100 can be calculated: the effective computing power required by a single server = computing power required for training / (number of servers*number of GPUs per server) = 8425926TFLOPs / (3500*8)≈300TFLOPs.

[0076] Based on the above steps, by allocating computing power to GPUs of different models, it is possible to flexibly adjust according to actual conditions to meet the needs of different data centers, avoid waste of resources, and ensure that each GPU can operate in its optimal state, thereby improving GPU utilization.

[0077] Step S2, calculating the target core frequency of the GPU according to the effective computing power that the GPU needs to provide.

[0078] In one possible implementation manner, the specific step of calculating the target core frequency of the GPU according to the effective computing power that the GPU needs to provide includes: calculating the target core frequency of the GPU by using a peak computing power calculation formula:

[0079] Peak FLOPS = F clk *N SM *T ins *2

[0080] Where PeakFLOPS is the effective computing power that the GPU needs to provide, F clk is the target core frequency of the GPU, N SM is the number of computing units of the GPU, T ins The instruction throughput for a specific data type.

[0081] Specifically, taking A100 GPU as an example, the number of computing units N is known. SMThe FP16 throughput is 108, and the FP16 throughput is 1024. Substituting the effective computing power of 300 TFLOPs that the GPU needs to provide calculated in step S1 into the peak computing power calculation formula, the target core frequency F of A100 can be calculated. clk It is 1.35GHz.

[0082] Based on the above steps, the target core frequency is calculated through the peak computing power formula to ensure the effective computing power that the GPU needs to provide when training large models, and optimize resource usage while meeting the required computing power to ensure that the GPU runs in the optimal state, thereby improving the heat dissipation performance of the entire GPU card, extending the card life, and improving the GPU energy efficiency ratio.

[0083] Step S3, periodically obtaining the current status information of the GPU, and determining whether the current status information exceeds a preset status threshold.

[0084] The GPU underlying hardware is composed of hundreds of billions of transistors, so the power consumption of the GPU is equal to the sum of all transistors. It is known that the power consumption in CMOS circuits includes static power consumption and dynamic power consumption, that is, P = ∑(CV2αf+VI dq ), where C is the transistor capacitance, V is the GPU core voltage, α is the flip rate at the current frequency, f is the GPU core frequency, and I dq is the static current. CV2, αf represents the dynamic power consumption, VI dq It represents static consumption. Therefore, it can be seen from the formula that if you want to reduce dynamic power consumption, you can start by adjusting the GPU core voltage V and GPU core frequency f.

[0085] In one possible implementation manner, the specific steps of periodically obtaining the current state information of the GPU and determining whether the current state information exceeds a preset state threshold include: obtaining a core voltage value range of the GPU, and calculating a core frequency dynamic change rate of the GPU according to a difference between a maximum value and a minimum value of the core voltage value range; obtaining a maximum core frequency of the GPU, and calculating a core frequency value range of the GPU according to the core frequency dynamic change rate and the maximum core frequency; periodically obtaining a current temperature of the GPU, and comparing the current temperature with a preset first temperature threshold and a second temperature threshold; in response to the current temperature being less than the first temperature threshold, taking the maximum value of the core voltage value range of the GPU as a target core voltage of the GPU; in response to the current temperature being greater than the second temperature threshold, taking the minimum value of the core voltage value range of the GPU as a target core voltage of the GPU; in response to the current temperature being less than or equal to the second temperature threshold and greater than or equal to the first temperature threshold, obtaining the current core frequency of the GPU, and if the current core frequency is within the core frequency value range of the GPU, determining that the difference between the current core frequency of the GPU and the target core frequency satisfies a preset range.

[0086] Exemplarily, the stable core voltage range of A100 GPU is 0.9V-1.1V, that is, the GPU core voltage can be reduced to 0.9V at most and increased to 1.1V at most. The difference between the maximum and minimum values ​​of its value range is ΔV=1.1V-0.9V=0.2V. According to the formula core voltage V=maximum core voltage*(1-k(f)), substituting the core voltage value range into the formula, the k(f) value range is 0≤k(f)≤18.1%, where k(f) represents the dynamic change rate of GPU core frequency, which is used to measure the degree of change of GPU core frequency relative to boost frequency (i.e. maximum core frequency), i.e. k(f)=f (GPUboost) -f (GPU核心频率) / f (GPUboost) , the maximum core frequency f of A100 is known (GPUboost) is 1.41GHz, so we can get the GPU core frequency range is 1.15GHz≤f (GPU核心频率) ≤1.41GHz.

[0087] For the GPU, high temperature may cause the core frequency to automatically reduce to prevent overheating. Therefore, the GPU periodically obtains the current temperature of the GPU during the training / inference task, for example, once every 200ms. When the current temperature of the GPU is less than the first temperature threshold (for example, 85°C), it means that the core frequency of the GPU is high at this time, and it can safely run at a higher voltage to support higher performance. Therefore, the operating voltage of the GPU is set to 1.1V at this time to ensure that the GPU can provide sufficient computing power; when the current temperature of the GPU is greater than the second temperature threshold (for example, 92°C), in order to prevent overheating, the core voltage can be reduced to reduce power consumption and heat generation, so the operating voltage is set to 0.9V to reduce the temperature to protect the GPU; when the current temperature of the GPU is between the first temperature threshold and the second temperature threshold (for example, 85-92°C), it is in a relatively safe operating range. At this time, the operating voltage can be calculated according to the target core frequency of the GPU, so that the GPU can maintain good performance without overheating.

[0088] Further, in response to the current temperature being less than or equal to the second temperature threshold and greater than or equal to the first temperature threshold, if the current core frequency does not meet the core frequency value range of the GPU, the current power consumption of the GPU is obtained; it is determined whether the current power consumption of the GPU is greater than a first preset power consumption threshold, and if so, the operating voltage of the GPU is reduced; the target power consumption of the GPU is determined according to the current temperature of the GPU, for example, the current temperature of the GPU can be amplified by a preset multiple and then filtered to obtain a processed current temperature, the processed current temperature is input into a proportional-integral-differential controller, and the target power consumption of the GPU is output; the second target core frequency and the second target core voltage of the GPU are determined according to the target power consumption and current power consumption of the GPU; the operating voltage of the GPU is adjusted according to the second target core voltage until the current core frequency meets the core frequency value range of the GPU.

[0089] Based on the above steps, the core voltage of the GPU can be adjusted according to its current temperature, thereby ensuring performance while taking into account energy consumption and temperature control, and ensuring that the GPU can operate in an optimal state under different load conditions to prevent system instability or hardware damage caused by overheating.

[0090] Step S4, in response to the current state information exceeding a preset state threshold, and the difference between the current core frequency of the GPU and the target core frequency satisfies a preset range, the target core voltage of the GPU is determined based on the target core frequency, and the operating voltage of the GPU is adjusted according to the target core frequency and the target core voltage.

[0091] In one possible implementation, the specific steps of determining the target core voltage of the GPU based on the target core frequency and adjusting the operating voltage of the GPU according to the target core frequency and the target core voltage include: determining the target core voltage corresponding to the target core frequency based on the relationship between the core frequency and the core voltage of the GPU; and setting the operating voltage of the GPU according to the target core voltage via the SMBus bus.

[0092] For example, the operating frequency of the GPU is adjusted to the target core frequency of 1.35 GHz. At this time, the dynamic change rate of the GPU core frequency k(f) is 4.25%. Therefore, the target core voltage can be calculated to be 1.053 V. The out-of-band GPU core voltage is set through the SMBus bus. The BMC serial port command is used to set it according to the following steps:

[0093] (1) Log in to the BMC serial port and stop GPU polling

[0094] #cd / etc / init.d&&. / procmanager stop&&. / updatemgr.sh stop

[0095] (2) Switch to the I2C channel where the GPU A100 is to be set

[0096] #i2ctransfer-y 12w3@0x310x810x850x1f; i2ctransfer-y 12w2@0x700x000x04;

[0097] (3) Use the SMBus bus to set the GPU core voltage. The steps are as follows:

[0098] #i2ctransfer-y 12w6@0x4f0x5d 0x040x010x000x000x00 r5;

[0099] #i2ctransfer-y 12w6@0x4f0x5c 0x040x0e 0x000x000x80 r5;

[0100] #i2ctransfer-y 12w1@0x4f0x5c r5;

[0101] #i2ctransfer-y 12w6@0x4f0x5d 0x040x1d 0x040x00x0 r5;

[0102] #i2ctransfer-y 12w6@0x4f0x5c 0x040x0e 0x010x000x80 r5;

[0103] #i2ctransfer-y 12w1@0x4f0x5c r5;

[0104] #i2ctransfer-y 12w6@0x4f0x5c 0x040x100x010x000x80 r5;

[0105] #i2ctransfer-y 12w1@0x4f0x5c r5;

[0106] #i2ctransfer-y 12w1@0x4f0x5d r5;

[0107] #i2ctransfer-y 12w6@0x4f0x5c 0x040x100xFF 0x000x80 r5;

[0108] #i2ctransfer-y 12w1@0x4f0x5c r5;

[0109] Among them, the SMBus register includes a command register and a data register, which follows the SMBus protocol to allow the system BMC to communicate with the GPU. The GPU core voltage can be adjusted within milliseconds through the power management chip.

[0110] Based on the above steps, an out-of-band method is used to dynamically adjust the GPU core voltage to reduce GPU power consumption, thereby reducing the total power of the entire system, aiming to meet the customer's computing power needs while meeting the heat dissipation needs of the entire machine, avoiding excessive power consumption, improving energy conversion efficiency, and achieving green and low-carbon development of data centers. Compared with the method of adjusting the graphics card voltage by modifying the graphics card VBIOS file, this method does not require physical contact with the server during setting, which improves work efficiency and does not affect the performance and stability of the server.

[0111] It should be understood that although Figure 1-3 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-3At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0112] In one embodiment, Figure 5 As shown, a GPU power consumption control device is provided, including: a first calculation module, a second calculation module, a judgment module and a setting module, wherein:

[0113] A first computing module is used to obtain parameter information and GPU resource information of the target model, and calculate the effective computing power that each GPU needs to provide according to the parameter information and GPU resource information;

[0114] A second calculation module, used for calculating the target core frequency of the GPU according to the effective computing power that the GPU needs to provide;

[0115] A determination module, used for periodically acquiring current status information of the GPU and determining whether the current status information exceeds a preset status threshold;

[0116] A setting module is used to determine a target core voltage of the GPU based on the target core frequency in response to the current state information exceeding a preset state threshold and the difference between the current core frequency of the GPU and the target core frequency satisfies a preset range, and adjust the operating voltage of the GPU according to the target core frequency and the target core voltage.

[0117] In one embodiment, a GPU scheduling module is also included, which is used to determine application scenarios according to business needs, and the application scenarios include model training, model reasoning, high-performance computing and image processing; according to the application scenarios, GPU resources are scheduled based on disaster recovery strategies.

[0118] In one embodiment, the first computing module is also used to calculate the number of floating-point operations required for training based on the model parameter quantity and the data set size; calculate the computing power required for training based on the number of floating-point operations required for training and the preset training time; based on the computing power required for training, calculate the effective computing power that each GPU needs to provide according to the GPU resource information.

[0119] In one embodiment, the first computing module is also used to determine whether the GPUs in the current server are of the same model; if so, the effective computing power that each GPU needs to provide is calculated based on the computing power required for the training and the number of GPUs; if not, the effective computing power that each GPU needs to provide is calculated based on the computing power required for the training and the computing power weights corresponding to each pre-configured GPU model.

[0120] In one embodiment, the second calculation module is further used to calculate the target core frequency of the GPU by using the peak computing power calculation formula: Peak FLOPS = F clk *N SM *T ins *2; PeakFLOPS is the effective computing power that the GPU needs to provide, F clk is the target core frequency of the GPU, N SM is the number of computing units of the GPU, T ins The instruction throughput for a specific data type.

[0121] In one embodiment, the judgment module is further used to obtain a core voltage value range of the GPU, and calculate a core frequency dynamic change rate of the GPU according to a difference between a maximum value and a minimum value of the core voltage value range;

[0122] The maximum core frequency of the GPU is obtained, and the core frequency value range of the GPU is calculated according to the core frequency dynamic change rate and the maximum core frequency; the current temperature of the GPU is periodically obtained, and the current temperature is compared with a preset first temperature threshold and a second temperature threshold; in response to the current temperature being less than the first temperature threshold, the maximum value of the core voltage value range of the GPU is used as the target core voltage of the GPU; in response to the current temperature being greater than the second temperature threshold, the minimum value of the core voltage value range of the GPU is used as the target core voltage of the GPU; in response to the current temperature being less than or equal to the second temperature threshold and greater than or equal to the first temperature threshold, the current core frequency of the GPU is obtained, and if the current core frequency is within the core frequency value range of the GPU, it is determined that the difference between the current core frequency of the GPU and the target core frequency meets the preset range.

[0123] In one embodiment, the setting module is further used to determine the target core voltage corresponding to the target core frequency based on the correspondence between the core frequency and the core voltage of the GPU; and to set the operating voltage of the GPU according to the target core voltage via the SMBus bus.

[0124] For the specific definition of the GPU power consumption control device, please refer to the definition of the GPU power consumption control method above, which will not be repeated here. Each module in the above-mentioned GPU power consumption control device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0125] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store GPU status data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a GPU power consumption control method is implemented.

[0126] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0127] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0128] Obtain parameter information and GPU resource information of the target model, and calculate the effective computing power that each GPU needs to provide based on the parameter information and GPU resource information;

[0129] Calculating the target core frequency of the GPU according to the effective computing power that the GPU needs to provide;

[0130] Periodically obtaining current status information of the GPU, and determining whether the current status information exceeds a preset status threshold;

[0131] In response to the current state information exceeding a preset state threshold, and the difference between the current core frequency of the GPU and the target core frequency satisfies a preset range, the target core voltage of the GPU is determined based on the target core frequency, and the operating voltage of the GPU is adjusted according to the target core frequency and the target core voltage.

[0132] In one embodiment, when the processor executes the computer program, the following steps are also implemented: determining application scenarios according to business needs, wherein the application scenarios include model training, model reasoning, high-performance computing, and image processing; and scheduling GPU resources based on the disaster recovery strategy according to the application scenarios.

[0133] In one embodiment, when the processor executes the computer program, the following steps are also implemented: the number of floating-point operations required for training is calculated based on the model parameter quantity and the data set size; the computing power required for training is calculated based on the number of floating-point operations required for training and the preset training time; based on the computing power required for training, the effective computing power that each GPU needs to provide is calculated according to the GPU resource information.

[0134] In one embodiment, when the processor executes the computer program, the following steps are also implemented: determining whether the GPUs in the current server are of the same model; if so, calculating the effective computing power that each GPU needs to provide based on the computing power required for the training and the number of GPUs; if not, calculating the effective computing power that each GPU needs to provide based on the computing power required for the training and the computing power weights corresponding to the pre-configured GPU models.

[0135] In one embodiment, when the processor executes the computer program, the following steps are further implemented: Calculate the target core frequency of the GPU by using the peak computing power calculation formula: Peak FLOPS = F clk *N SM *T ins *2; PeakFLOPS is the effective computing power that the GPU needs to provide, F clk is the target core frequency of the GPU, N SM is the number of computing units of the GPU, T ins The instruction throughput for a specific data type.

[0136] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining the core voltage value range of the GPU, and calculating the core frequency dynamic change rate of the GPU according to the difference between the maximum value and the minimum value of the core voltage value range; obtaining the maximum core frequency of the GPU, and calculating the core frequency value range of the GPU according to the core frequency dynamic change rate and the maximum core frequency; periodically obtaining the current temperature of the GPU, and comparing the current temperature with a preset first temperature threshold and a second temperature threshold; in response to the current temperature being less than the first temperature threshold, taking the maximum value of the core voltage value range of the GPU as the target core voltage of the GPU; in response to the current temperature being greater than the second temperature threshold, taking the minimum value of the core voltage value range of the GPU as the target core voltage of the GPU; in response to the current temperature being less than or equal to the second temperature threshold and greater than or equal to the first temperature threshold, obtaining the current core frequency of the GPU, and if the current core frequency is within the core frequency value range of the GPU, determining that the difference between the current core frequency of the GPU and the target core frequency meets the preset range.

[0137] In one embodiment, when the processor executes the computer program, the following steps are also implemented: based on the correspondence between the core frequency and the core voltage of the GPU, determining the target core voltage corresponding to the target core frequency; and setting the operating voltage of the GPU according to the target core voltage through the SMBus bus.

[0138] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0139] Obtain parameter information and GPU resource information of the target model, and calculate the effective computing power that each GPU needs to provide based on the parameter information and GPU resource information;

[0140] Calculating the target core frequency of the GPU according to the effective computing power that the GPU needs to provide;

[0141] Periodically obtaining current status information of the GPU, and determining whether the current status information exceeds a preset status threshold;

[0142] In response to the current state information exceeding a preset state threshold, and the difference between the current core frequency of the GPU and the target core frequency satisfies a preset range, the target core voltage of the GPU is determined based on the target core frequency, and the operating voltage of the GPU is adjusted according to the target core frequency and the target core voltage.

[0143] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining application scenarios according to business needs, wherein the application scenarios include model training, model reasoning, high-performance computing, and image processing; and scheduling GPU resources based on the disaster recovery strategy according to the application scenarios.

[0144] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: the number of floating-point operations required for training is calculated based on the model parameter quantity and the data set size; the computing power required for training is calculated based on the number of floating-point operations required for training and the preset training time; based on the computing power required for training, the effective computing power that each GPU needs to provide is calculated according to the GPU resource information.

[0145] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining whether the GPUs in the current server are of the same model; if so, calculating the effective computing power that each GPU needs to provide based on the computing power required for the training and the number of GPUs; if not, calculating the effective computing power that each GPU needs to provide based on the computing power required for the training and the computing power weights corresponding to the pre-configured GPU models.

[0146] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: Calculate the target core frequency of the GPU by using the peak computing power calculation formula: Peak FLOPS = F clk *N SM *T ins *2; PeakFLOPS is the effective computing power that the GPU needs to provide, F clk is the target core frequency of the GPU, N SM is the number of computing units of the GPU, T ins The instruction throughput for a specific data type.

[0147] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining the core voltage value range of the GPU, and calculating the core frequency dynamic change rate of the GPU according to the difference between the maximum value and the minimum value of the core voltage value range; obtaining the maximum core frequency of the GPU, and calculating the core frequency value range of the GPU according to the core frequency dynamic change rate and the maximum core frequency; periodically obtaining the current temperature of the GPU, and comparing the current temperature with a preset first temperature threshold and a second temperature threshold; in response to the current temperature being less than the first temperature threshold, taking the maximum value of the core voltage value range of the GPU as the target core voltage of the GPU; in response to the current temperature being greater than the second temperature threshold, taking the minimum value of the core voltage value range of the GPU as the target core voltage of the GPU; in response to the current temperature being less than or equal to the second temperature threshold and greater than or equal to the first temperature threshold, obtaining the current core frequency of the GPU, and if the current core frequency is within the core frequency value range of the GPU, determining that the difference between the current core frequency of the GPU and the target core frequency meets the preset range.

[0148] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on the correspondence between the core frequency and the core voltage of the GPU, determining the target core voltage corresponding to the target core frequency; and setting the operating voltage of the GPU according to the target core voltage via the SMBus bus.

[0149] In one embodiment, a computer product is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0150] Obtain parameter information and GPU resource information of the target model, and calculate the effective computing power that each GPU needs to provide based on the parameter information and GPU resource information;

[0151] Calculating the target core frequency of the GPU according to the effective computing power that the GPU needs to provide;

[0152] Periodically obtaining current status information of the GPU, and determining whether the current status information exceeds a preset status threshold;

[0153] In response to the current state information exceeding a preset state threshold, and the difference between the current core frequency of the GPU and the target core frequency satisfies a preset range, the target core voltage of the GPU is determined based on the target core frequency, and the operating voltage of the GPU is adjusted according to the target core frequency and the target core voltage.

[0154] In one embodiment, when the processor executes the computer program, the following steps are also implemented: determining application scenarios according to business needs, wherein the application scenarios include model training, model reasoning, high-performance computing, and image processing; and scheduling GPU resources based on the disaster recovery strategy according to the application scenarios.

[0155] In one embodiment, when the processor executes the computer program, the following steps are also implemented: the number of floating-point operations required for training is calculated based on the model parameter quantity and the data set size; the computing power required for training is calculated based on the number of floating-point operations required for training and the preset training time; based on the computing power required for training, the effective computing power that each GPU needs to provide is calculated according to the GPU resource information.

[0156] In one embodiment, when the processor executes the computer program, the following steps are also implemented: determining whether the GPUs in the current server are of the same model; if so, calculating the effective computing power that each GPU needs to provide based on the computing power required for the training and the number of GPUs; if not, calculating the effective computing power that each GPU needs to provide based on the computing power required for the training and the computing power weights corresponding to the pre-configured GPU models.

[0157] In one embodiment, when the processor executes the computer program, the following steps are further implemented: Calculate the target core frequency of the GPU by using the peak computing power calculation formula: Peak FLOPS = F clk *N SM *T ins *2; PeakFLOPS is the effective computing power that the GPU needs to provide, F clk is the target core frequency of the GPU, N SM is the number of computing units of the GPU, T ins The instruction throughput for a specific data type.

[0158] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining the core voltage value range of the GPU, and calculating the core frequency dynamic change rate of the GPU according to the difference between the maximum value and the minimum value of the core voltage value range; obtaining the maximum core frequency of the GPU, and calculating the core frequency value range of the GPU according to the core frequency dynamic change rate and the maximum core frequency; periodically obtaining the current temperature of the GPU, and comparing the current temperature with a preset first temperature threshold and a second temperature threshold; in response to the current temperature being less than the first temperature threshold, taking the maximum value of the core voltage value range of the GPU as the target core voltage of the GPU; in response to the current temperature being greater than the second temperature threshold, taking the minimum value of the core voltage value range of the GPU as the target core voltage of the GPU; in response to the current temperature being less than or equal to the second temperature threshold and greater than or equal to the first temperature threshold, obtaining the current core frequency of the GPU, and if the current core frequency is within the core frequency value range of the GPU, determining that the difference between the current core frequency of the GPU and the target core frequency meets the preset range.

[0159] In one embodiment, when the processor executes the computer program, the following steps are also implemented: based on the correspondence between the core frequency and the core voltage of the GPU, determining the target core voltage corresponding to the target core frequency; and setting the operating voltage of the GPU according to the target core voltage through the SMBus bus.

[0160] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0161] Obtain parameter information and GPU resource information of the target model, and calculate the effective computing power that each GPU needs to provide based on the parameter information and GPU resource information;

[0162] Calculating the target core frequency of the GPU according to the effective computing power that the GPU needs to provide;

[0163] Periodically obtaining current status information of the GPU, and determining whether the current status information exceeds a preset status threshold;

[0164] In response to the current state information exceeding a preset state threshold, and the difference between the current core frequency of the GPU and the target core frequency satisfies a preset range, the target core voltage of the GPU is determined based on the target core frequency, and the operating voltage of the GPU is adjusted according to the target core frequency and the target core voltage.

[0165] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining application scenarios according to business needs, wherein the application scenarios include model training, model reasoning, high-performance computing, and image processing; and scheduling GPU resources based on the disaster recovery strategy according to the application scenarios.

[0166] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: the number of floating-point operations required for training is calculated based on the model parameter quantity and the data set size; the computing power required for training is calculated based on the number of floating-point operations required for training and the preset training time; based on the computing power required for training, the effective computing power that each GPU needs to provide is calculated according to the GPU resource information.

[0167] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining whether the GPUs in the current server are of the same model; if so, calculating the effective computing power that each GPU needs to provide based on the computing power required for the training and the number of GPUs; if not, calculating the effective computing power that each GPU needs to provide based on the computing power required for the training and the computing power weights corresponding to the pre-configured GPU models.

[0168] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: Calculate the target core frequency of the GPU by using the peak computing power calculation formula: Peak FLOPS = F clk *N SM *T ins *2; PeakFLOPS is the effective computing power that the GPU needs to provide, F clk is the target core frequency of the GPU, N SM is the number of computing units of the GPU, T ins The instruction throughput for a specific data type.

[0169] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining the core voltage value range of the GPU, and calculating the core frequency dynamic change rate of the GPU according to the difference between the maximum value and the minimum value of the core voltage value range; obtaining the maximum core frequency of the GPU, and calculating the core frequency value range of the GPU according to the core frequency dynamic change rate and the maximum core frequency; periodically obtaining the current temperature of the GPU, and comparing the current temperature with a preset first temperature threshold and a second temperature threshold; in response to the current temperature being less than the first temperature threshold, taking the maximum value of the core voltage value range of the GPU as the target core voltage of the GPU; in response to the current temperature being greater than the second temperature threshold, taking the minimum value of the core voltage value range of the GPU as the target core voltage of the GPU; in response to the current temperature being less than or equal to the second temperature threshold and greater than or equal to the first temperature threshold, obtaining the current core frequency of the GPU, and if the current core frequency is within the core frequency value range of the GPU, determining that the difference between the current core frequency of the GPU and the target core frequency meets the preset range.

[0170] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on the correspondence between the core frequency and the core voltage of the GPU, determining the target core voltage corresponding to the target core frequency; and setting the operating voltage of the GPU according to the target core voltage via the SMBus bus.

[0171] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0172] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0173] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A GPU power consumption control method, characterized in that: include: Obtain parameter information and GPU resource information of the target model, and calculate the effective computing power that each GPU needs to provide based on the parameter information and GPU resource information; Calculating the target core frequency of the GPU according to the effective computing power that the GPU needs to provide; Periodically obtaining current status information of the GPU, and determining whether the current status information exceeds a preset status threshold; In response to the current state information exceeding a preset state threshold, and the difference between the current core frequency of the GPU and the target core frequency satisfies a preset range, the target core voltage of the GPU is determined based on the target core frequency, and the operating voltage of the GPU is adjusted according to the target core frequency and the target core voltage.

2. The GPU power consumption control method according to claim 1, characterized in that: Before obtaining the parameter information and GPU resource information of the target model, the method further includes: Determine application scenarios based on business needs, including model training, model reasoning, high-performance computing, and image processing; According to the application scenario, GPU resources are scheduled based on the disaster recovery strategy.

3. The GPU power consumption control method according to claim 1, characterized in that: The parameter information of the target model includes the model parameter quantity and the data set size. The acquiring the parameter information and GPU resource information of the target model, and calculating the effective computing power that the GPU needs to provide according to the parameter information and GPU resource information, includes: Calculate the number of floating-point operations required for training according to the model parameters and the size of the data set; Calculate the computing power required for training according to the number of floating-point operations required for the training and the preset training time; Based on the computing power required for the training, the effective computing power that each GPU needs to provide is calculated according to the GPU resource information.

4. The GPU power consumption control method according to claim 3, characterized in that: The GPU resource information includes the GPU model and quantity; the effective computing power required for training is calculated based on the GPU resource information, including: Determine whether the GPUs in the current server are of the same model; If yes, then calculate the effective computing power that each GPU needs to provide based on the computing power required for the training and the number of GPUs; If not, the effective computing power that each GPU needs to provide is calculated based on the computing power required for the training and the computing power weights corresponding to the pre-configured GPU models.

5. The GPU power consumption control method according to claim 1, characterized in that: The calculating the target core frequency of the GPU according to the effective computing power that the GPU needs to provide includes: The target core frequency of the GPU is calculated using the peak computing power calculation formula: PeakFLOPS=F clk *N SM *T ins *2; Where PeakFLOPS is the effective computing power that the GPU needs to provide, F clk is the target core frequency of the GPU, N SM is the number of computing units of the GPU, T ins The instruction throughput for a specific data type.

6. The GPU power consumption control method according to claim 1, characterized in that: The periodically acquiring the current state information of the GPU and determining whether the current state information exceeds a preset state threshold includes: Obtaining a core voltage value range of the GPU, and calculating a core frequency dynamic change rate of the GPU according to a difference between a maximum value and a minimum value of the core voltage value range; Obtaining the maximum core frequency of the GPU, and calculating the core frequency value range of the GPU according to the core frequency dynamic change rate and the maximum core frequency; Periodically acquiring a current temperature of the GPU, and comparing the current temperature with a preset first temperature threshold and a second temperature threshold; In response to the current temperature being less than the first temperature threshold, taking a maximum value of a core voltage value range of the GPU as a target core voltage of the GPU; In response to the current temperature being greater than the second temperature threshold, taking a minimum value of a core voltage value range of the GPU as a target core voltage of the GPU; In response to the current temperature being less than or equal to the second temperature threshold and greater than or equal to the first temperature threshold, the current core frequency of the GPU is obtained; if the current core frequency is within the core frequency value range of the GPU, it is determined that the difference between the current core frequency of the GPU and the target core frequency satisfies a preset range.

7. The GPU power consumption control method according to claim 1, characterized in that: The step of determining a target core voltage of the GPU based on the target core frequency, and adjusting an operating voltage of the GPU according to the target core frequency and the target core voltage includes: Determining a target core voltage corresponding to the target core frequency based on a correspondence between a core frequency and a core voltage of the GPU; The operating voltage of the GPU is set according to the target core voltage through the SMBus bus.

8. A GPU power consumption control device, characterized in that: The device comprises: A first computing module is used to obtain parameter information and GPU resource information of the target model, and calculate the effective computing power that each GPU needs to provide according to the parameter information and GPU resource information; A second calculation module, used for calculating the target core frequency of the GPU according to the effective computing power that the GPU needs to provide; A determination module, used for periodically acquiring current status information of the GPU and determining whether the current status information exceeds a preset status threshold; A setting module is used to determine a target core voltage of the GPU based on the target core frequency in response to the current state information exceeding a preset state threshold and the difference between the current core frequency of the GPU and the target core frequency satisfies a preset range, and adjust the operating voltage of the GPU according to the target core frequency and the target core voltage.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images