An energy efficiency optimization method based on dynamic voltage and frequency adjustment

Abstract

The present invention discloses an energy efficiency optimization method based on dynamic voltage and frequency adjustment, comprising: in an offline state of a large language model, performing energy consumption tests on data of different load sizes to obtain energy consumption performance under different dynamic voltage and frequency adjustment configurations; determining the optimal dynamic voltage and frequency adjustment configurations for each batch in a pre-filling stage and a decoding stage according to the obtained energy consumption performance under the different dynamic voltage and frequency adjustment configurations; adjusting the core frequency and memory frequency of a graphics processor according to the determined optimal dynamic voltage and frequency adjustment configuration to optimize the energy efficiency of large language model reasoning; the present invention proposes an optimization scheme based on dynamic voltage and frequency adjustment (DVFS), and for the pre-filling stage and the decoding stage of large model reasoning, different dynamic voltage and frequency adjustment configurations are respectively applied to achieve energy consumption optimization.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an energy efficiency optimization method based on dynamic voltage and frequency adjustment. Background Art

[0002] The reasoning process of large language models (LLMs) involves a lot of computation, and these computational operations consume a lot of electricity, especially when running on specialized hardware such as GPUs (graphics processing units). As the size of the model continues to increase, its energy consumption has also shown a significant growth trend, which not only increases the operating cost, but also aggravates the impact on the ambient temperature. Therefore, optimizing the energy consumption of these models can not only reduce costs, but also help reduce carbon footprint and achieve more sustainable AI technology applications. To address this problem, dynamic voltage frequency scaling (DVFS) has become a promising technology that can adjust devices to lower performance / power states. DVFS optimizes GPU performance and power consumption by adjusting voltage and frequency levels, providing considerable energy savings with little impact on performance, saving energy while maintaining large model inference performance.

[0003] Existing large model inference energy consumption optimization technologies mainly focus on model compression, hardware optimization, and software and algorithm improvements. These technologies each have their own advantages, but compared with the stage-separated dynamic voltage and frequency scaling (DVFS) technology, there are some potential disadvantages:

[0004] 1) Model compression techniques such as pruning, knowledge distillation, and quantization can reduce model size and reduce the amount of computation during inference, thereby reducing energy consumption. However, these techniques may sacrifice model accuracy or performance. Compared with DVFS technology, model compression usually needs to be performed before model deployment, which may limit the model's ability to dynamically adjust performance and energy consumption at runtime.

[0005] 2) Hardware optimizations, such as custom accelerators or efficient data center designs, can significantly improve energy efficiency, but these optimizations often require expensive investments and are not necessarily flexible enough to adapt to different workloads.

[0006] 3) Software-level optimizations, such as efficient inference engines and algorithm improvements, can reduce energy consumption. However, these optimizations usually rely on specific algorithms or model architectures and may not be universally applicable.

[0007] In general, although existing large model inference energy consumption optimization techniques play an important role in reducing energy consumption, they usually lack the flexibility and wide applicability of real-time and dynamic energy consumption adjustment provided by DVFS.

[0008] Therefore, the prior art needs to be improved. Summary of the invention

[0009] The technical problem to be solved by the present invention is that, in view of the defects of the prior art, the present invention provides an energy efficiency optimization method based on dynamic voltage and frequency adjustment to solve the problem that the existing large model reasoning energy consumption optimization technology lacks the flexibility and wide applicability of real-time and dynamic adjustment of energy consumption.

[0010] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0011] In a first aspect, the present invention provides an energy efficiency optimization method based on dynamic voltage and frequency adjustment, comprising:

[0012] In the offline state of a large language model, energy consumption tests are performed on data of different load sizes to obtain energy consumption performance under different dynamic voltage and frequency adjustment configurations;

[0013] Determine the dynamic voltage and frequency adjustment configuration of each batch in the pre-filling stage and the decoding stage according to the obtained energy consumption performance under different dynamic voltage and frequency adjustment configurations;

[0014] The GPU core frequency and memory frequency are adjusted according to the determined dynamic voltage-frequency scaling configuration to optimize the energy efficiency of large language model inference.

[0015] In one implementation, the energy consumption test is performed on data of different load sizes in an offline state of a large language model to obtain energy consumption performance under different dynamic voltage and frequency adjustment configurations, including:

[0016] In the offline state of a large language model, based on the power consumption test tool provided by the GPU management library, the following formula is used to test the energy consumption of data with different load sizes:

[0017] ;

[0018] ;

[0019] ;

[0020] in, Indicates load; Indicates the graphics processor core frequency; Indicates the graphics processor memory frequency;

[0021] , , Respectively represent the batch size, input length, and output length of the input data;

[0022] , , They represent the energy consumption, power and delay values ​​in the pre-filling phase respectively;

[0023] , , They represent the energy consumption, power and delay values ​​in the decoding stage respectively.

[0024] In one implementation, the power of the pre-filling stage and the power of the decoding stage are both average powers of the corresponding stages measured and calculated by a power consumption test tool provided by the graphics processor management library; the delay value of the pre-filling stage and the delay value of the decoding stage are both time differences between the beginning and the end of the corresponding stages.

[0025] In one implementation, determining the dynamic voltage and frequency adjustment configuration of each batch in the pre-filling stage and the decoding stage according to the obtained energy consumption performance under different dynamic voltage and frequency adjustment configurations includes:

[0026] According to the calculated energy consumption of the pre-filling stage and the energy consumption of the decoding stage, the dynamic voltage and frequency adjustment configuration of different loads in the corresponding pre-filling stage and decoding stage is calculated using the following formula:

[0027] ;

[0028] ;

[0029] in, represents the dynamic voltage and frequency adjustment configuration in the pre-filling stage; Indicates the dynamic voltage and frequency adjustment configuration at the decoding stage.

[0030] In one implementation, the method of adjusting the graphics processor core frequency and memory frequency according to the determined dynamic voltage and frequency adjustment configuration to optimize the energy efficiency of large language model reasoning includes:

[0031] When the large language model is in an online state, it is dynamically determined whether a dynamic voltage and frequency adjustment optimization strategy needs to be implemented in the current stage according to the batch size, input length and corresponding stage of the online reasoning of the large language model.

[0032] In one implementation, in the online state of the large language model, dynamically determining whether a dynamic voltage and frequency adjustment optimization strategy needs to be implemented in the current stage according to the batch size, input length and corresponding stage of the online reasoning of the large language model includes:

[0033] For the inference request of the pre-filling stage, the delay value of executing the pre-filling stage once is determined according to the test result in the offline state, and the delay value is compared with the preset frequency and voltage modulation threshold. Performing a comparison, and determining, based on the comparison result, whether the dynamic voltage and frequency adjustment optimization strategy needs to be implemented at the current stage;

[0034] For the inference request in the decoding stage, the duration from the last dynamic voltage and frequency adjustment optimization in the current stage is calculated based on the global timer, and the calculated duration is compared with the preset adjustment interval. A comparison is performed, and according to the comparison result, it is determined that the dynamic voltage and frequency adjustment optimization strategy needs to be implemented in the current stage.

[0035] In one implementation, adjusting the graphics processor core frequency and memory frequency according to the determined dynamic voltage frequency adjustment configuration includes:

[0036] According to the determined dynamic voltage and frequency adjustment configuration, the voltage and frequency targets required by the graphics processor are set using an application program interface provided by a graphics processor management library, and the graphics processor core frequency and the memory frequency are adjusted.

[0037] In a second aspect, the present invention provides an energy efficiency optimization device based on dynamic voltage and frequency adjustment, comprising:

[0038] The energy consumption test module is used to perform energy consumption tests on data of different load sizes in the offline state of a large language model, and obtain the energy consumption performance under different dynamic voltage and frequency adjustment configurations;

[0039] A dynamic voltage and frequency adjustment configuration module is used to determine the dynamic voltage and frequency adjustment configuration of each batch in the pre-filling stage and the decoding stage according to the energy consumption performance under different dynamic voltage and frequency adjustment configurations obtained;

[0040] The energy efficiency optimization module is used to adjust the graphics processor core frequency and memory frequency according to the determined dynamic voltage and frequency adjustment configuration to optimize the energy efficiency of large language model inference.

[0041] In a third aspect, the present invention provides a terminal, comprising: a processor and a memory, wherein the memory stores an energy efficiency optimization program based on dynamic voltage and frequency adjustment, and when the energy efficiency optimization program based on dynamic voltage and frequency adjustment is executed by the processor, it is used to implement the operation of the energy efficiency optimization method based on dynamic voltage and frequency adjustment as described in the first aspect.

[0042] In a fourth aspect, the present invention further provides a medium, which is a computer-readable storage medium, and which stores an energy efficiency optimization program based on dynamic voltage and frequency adjustment. When the energy efficiency optimization program based on dynamic voltage and frequency adjustment is executed by a processor, it is used to implement the operation of the energy efficiency optimization method based on dynamic voltage and frequency adjustment as described in the first aspect.

[0043] The present invention adopts the above technical solution to achieve the following effects:

[0044] The present invention performs energy consumption tests on data of different load sizes in an offline state of a large language model to obtain energy consumption performance under different dynamic voltage and frequency adjustment configurations; the optimal dynamic voltage and frequency adjustment configurations of each batch in the pre-filling stage and the decoding stage can be determined according to the obtained energy consumption performance under the different dynamic voltage and frequency adjustment configurations; and the graphics processor core frequency and memory frequency are adjusted according to the determined optimal dynamic voltage and frequency adjustment configuration to optimize the energy efficiency of large language model reasoning; the present invention collects power and delay data of the pre-filling stage and the decoding stage of the large model reasoning respectively, and optimizes them respectively, thereby achieving more refined optimization control and better energy efficiency optimization effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0046] Figure 1 It is a flow chart of the energy efficiency optimization method based on dynamic voltage and frequency adjustment in the present invention.

[0047] Figure 2 It is an overall schematic diagram of energy efficiency optimization based on dynamic voltage and frequency adjustment in the present invention.

[0048] Figure 3 It is a schematic diagram of part of the energy consumption of the large language model in the present invention under offline testing.

[0049] Figure 4 It is a schematic diagram of the online reasoning energy consumption optimization effect of the large language model in the present invention.

[0050] Figure 5 It is a functional principle diagram of a terminal in one implementation of the present invention.

[0051] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0053] Exemplary Methods

[0054] Existing large model reasoning energy consumption optimization technologies have some potential disadvantages: First, the use of model compression techniques (e.g., pruning, knowledge distillation, and quantization) can reduce the model size, reduce the amount of computation during reasoning, and thus reduce energy consumption. However, these techniques may sacrifice the accuracy or performance of the model. Compared with DVFS (dynamic voltage frequency scaling) technology, model compression usually needs to be performed before the model is deployed, which may limit the ability of the model to dynamically adjust performance and energy consumption at runtime. Second, the use of hardware optimization techniques (e.g., customized accelerators or efficient data center designs) can significantly improve energy efficiency, but these optimizations often require expensive investments and may not be flexible to different workloads. Finally, the use of software-level optimization techniques (e.g., efficient reasoning engines and algorithm improvements) can reduce energy consumption. However, these optimizations usually rely on specific algorithms or model architectures and may not be universally applicable.

[0055] In general, although existing large model inference energy consumption optimization techniques play an important role in reducing energy consumption, they usually lack the flexibility and wide applicability of real-time and dynamic energy consumption adjustment provided by DVFS.

[0056] In response to the above technical problems, an embodiment of the present invention provides an energy efficiency optimization method based on dynamic voltage and frequency adjustment. The method mainly performs energy consumption tests on data of different load sizes in an offline state of a large language model to obtain energy consumption performance under different dynamic voltage and frequency adjustment configurations; the optimal dynamic voltage and frequency adjustment configuration for each batch in the pre-filling stage and the decoding stage can be determined according to the obtained energy consumption performance under the different dynamic voltage and frequency adjustment configurations; and the GPU (graphics processing unit) core frequency and memory frequency are adjusted according to the determined optimal dynamic voltage and frequency adjustment configuration to optimize the energy efficiency of large language model reasoning; therefore, the embodiment of the present invention can collect power and delay data of the two stages respectively for the pre-filling stage and the decoding stage of the large model reasoning, and optimize them respectively, thereby achieving more refined optimization control and better energy efficiency optimization effect.

[0057] like Figure 1 As shown, an embodiment of the present invention provides an energy efficiency optimization method based on dynamic voltage and frequency adjustment, comprising the following steps:

[0058] Step S100 , in an offline state of a large language model, an energy consumption test is performed on data of different load sizes to obtain energy consumption performance under different dynamic voltage and frequency adjustment configurations.

[0059] In this embodiment, an optimization scheme based on dynamic voltage frequency scaling (DVFS) is proposed. Different dynamic voltage frequency scaling configurations are applied to the pre-filling stage and the decoding stage of large language model inference to achieve energy consumption optimization.

[0060] like Figure 2 As shown, in this embodiment, first, in the offline state of the large language model, the data of different load sizes (i.e., the batch size, input length, and output length of the large language model) are tested to measure the energy consumption performance under different DVFS configurations (dynamic voltage frequency adjustment configurations), so as to determine the optimal DVFS configuration for each batch in the pre-filling stage and the decoding stage according to the energy consumption performance; then, in the online reasoning scenario of the large language model, it is determined whether to implement dynamic voltage frequency adjustment optimization according to the load (such as batch size and input length) and different stages. When the delay in the pre-filling stage is greater than the threshold Or the decoding phase interval is greater than When the energy consumption data is measured offline, the energy consumption data of the actual load is obtained through secondary interpolation, and then the optimal DVFS configuration is solved. The GPU core frequency and memory frequency are adjusted according to the configuration to achieve energy consumption optimization.

[0061] Specifically, in an implementation of this embodiment, step S100 includes the following steps:

[0062] Step S101 , in an offline state of a large language model, an energy consumption test is performed on data of different load sizes based on a power consumption test tool provided by an NVML library (i.e., a graphics processor management library provided by a certain company).

[0063] In this embodiment, firstly, data with different load sizes (load) are tested in an offline state, and different DVFS configurations ( ) is shown in the following formula:

[0064] ;

[0065] ;

[0066] ;

[0067] in, Indicates load; Indicates GPU core frequency; Indicates GPU memory frequency; , , Respectively represent the batch size, input length, and output length of the input data; , , They represent the energy consumption, power and delay values ​​in the pre-filling phase respectively; , , They represent the energy consumption, power and delay values ​​in the decoding stage respectively.

[0068] In this embodiment, for the batch size, input length, and output length of the input data of the large language model inference process, the batch size refers to the number of data samples processed simultaneously during a single inference process; the input length refers to the size or length of a single input data accepted by the model; and the output length generally refers to the size or length of the output data generated by the model. and the power of the decoding stage , are the average power of the corresponding stage measured and calculated by the power consumption test tool provided by the NVML library; the specific calculation process is: use the power consumption test tool provided by the NVML library to measure the power once every certain time interval t, and finally get the average power of the corresponding stage according to the recorded log. The delay value of the pre-filling stage and the delay value of the decoding stage are both the time difference between the beginning and the end of the corresponding stage, that is, the delay value (Latency) only needs to be timed at the beginning and end of each stage, and then the difference can be obtained.

[0069] In this embodiment, through the above offline testing process of data with different load sizes, the power consumption data corresponding to the data with different load sizes in the pre-filling stage can be obtained. , and obtain the power consumption data corresponding to the data of different load sizes in the decoding stage According to the power consumption data corresponding to different loads at different stages, the optimal DVFS configuration of each batch in the pre-filling stage and the decoding stage can be calculated.

[0070] like Figure 1 As shown, in an implementation of an embodiment of the present invention, the energy efficiency optimization method based on dynamic voltage and frequency adjustment also includes the following steps:

[0071] Step S200 , determining the dynamic voltage and frequency adjustment configurations of each batch in the pre-filling stage and the decoding stage according to the obtained energy consumption performance under different dynamic voltage and frequency adjustment configurations.

[0072] In this embodiment, according to the power consumption data corresponding to the pre-filling stage And the power consumption data corresponding to the decoding stage , we can get the optimal DVFS configuration for different loads in the pre-filling stage and the decoding stage.

[0073] Specifically, in an implementation of this embodiment, step S200 includes the following steps:

[0074] Step S201 , according to the calculated energy consumption of the pre-filling stage and the energy consumption of the decoding stage, the dynamic voltage and frequency adjustment configurations of different loads in the corresponding pre-filling stage and decoding stage are calculated.

[0075] In this embodiment, the optimal DVFS configuration for different loads in the pre-filling stage and the decoding stage is calculated as follows:

[0076] ;

[0077] ;

[0078] in, Indicates the optimal DVFS configuration in the pre-population phase; Represents the optimal DVFS configuration in the decoding phase.

[0079] In this embodiment, the optimal DVFS configuration corresponding to the two stages (pre-filling stage and decoding stage) of the large language model in the offline state is calculated according to the above method; according to these optimal DVFS configurations in the offline state, the energy consumption of the GPU can be adjusted in real time and dynamically when the large language model is online.

[0080] like Figure 1 As shown, in an implementation of an embodiment of the present invention, the energy efficiency optimization method based on dynamic voltage and frequency adjustment also includes the following steps:

[0081] Step S300, adjusting the GPU core frequency and the memory frequency according to the determined dynamic voltage frequency adjustment configuration to optimize the energy efficiency of large language model reasoning.

[0082] It should be noted that in this embodiment, in the online reasoning scenario of a large language model, before dynamically optimizing the energy consumption of the GPU, it is also necessary to dynamically determine whether the current reasoning stage needs to implement a dynamic voltage and frequency adjustment optimization process based on the current load of the online reasoning (i.e., batch size and input length) and the corresponding stage.

[0083] Specifically, in an implementation of this embodiment, the following steps are included before step S300:

[0084] Step S301a, when the large language model is online, dynamically determine whether a dynamic voltage and frequency adjustment optimization strategy needs to be implemented in the current stage according to the batch size, input length and corresponding stage of the online reasoning of the large language model.

[0085] Specifically, in an implementation of this embodiment, when the large language model is online, according to the batch size, input length and corresponding stage of the online reasoning of the large language model, it is dynamically determined whether the current stage needs to implement the DVFS optimization strategy (i.e., dynamic voltage and frequency adjustment optimization strategy), including: for the reasoning request of the pre-filling stage, according to the test result in the offline state, the delay value of executing the pre-filling stage once is determined, and the delay value is compared with the preset frequency and voltage regulation threshold value. Compare and determine the need to implement the DVFS optimization strategy in the current stage according to the comparison result; for the inference request in the decoding stage, calculate the duration of the current stage from the last DVFS optimization based on the global timer, and compare the calculated duration with the preset adjustment interval A comparison is performed, and according to the comparison result, it is determined that the DVFS optimization strategy needs to be implemented in the current stage.

[0086] As an example, in this embodiment, in the process of dynamically determining whether to implement DVFS optimization, the basis for the determination is that for a request to perform a pre-filling phase, the delay value of executing a pre-filling phase is determined based on the test in the offline state, and then compared with the frequency and voltage threshold corresponding to the pre-filling phase. For comparison, if the delay value exceeds the threshold, DVFS optimization is required; for decoding stage requests, an adjustment interval is pre-set At the same time, there is a global timer, which is used to determine the time since the last DVFS optimization. When this interval is reached, DVFS optimization is required.

[0087] In the present invention, and The value of is an empirical value determined based on the system's preference for performance and energy efficiency. The smaller the value, the more emphasis is placed on the pre-filling stage performance, and vice versa, the more emphasis is placed on saving energy. The smaller the value, the more emphasis is placed on the performance of the decoding stage, and vice versa. and The setting is mainly based on actual application requirements. and value to achieve a trade-off between performance and energy consumption.

[0088] about and The following is an example of the value range:

[0089] The value range of is:

[0090] When the system has high performance requirements, Should be set to a larger value. For example, It can be between 0.5-1.0 to place more emphasis on the performance of the pre-filling stage.

[0091] If the system focuses more on energy efficiency, that is, it hopes to save more energy in the pre-filling stage, Can be set to a smaller value, ranging from 0.1-0.5.

[0092] The value range of is:

[0093] When the performance of the decoding stage is more critical, The value of should be larger. For example, It can be set between 0.5 and 1.0 to ensure high performance in the decoding stage.

[0094] On the contrary, if we prefer to save energy in the decoding stage, The value should be set to a small value, which can be in the range of 0.1-0.5.

[0095] Specifically, in an implementation of this embodiment, step S300 includes the following steps:

[0096] Step S301, according to the determined dynamic voltage and frequency adjustment configuration, using the API provided by the NVML library to set the voltage and frequency targets required by the GPU, and adjusting the GPU core frequency and the memory frequency.

[0097] In this embodiment, the specific optimization process is as follows: for the batch size and input length of the current scheduling request, this embodiment can find the corresponding bs1 (batch size 1), input-len1 (input length 1), bs2 (batch size 2), and input-len2 (input length 2) measured in the offline state near the load. There is no restriction on the selection of output length, because different output lengths show the same DVFS characteristics when other conditions are the same. Therefore, in this embodiment, the energy consumption corresponding to the current load can be obtained by secondary interpolation:

[0098] ;

[0099] ;

[0100] ;

[0101] In this embodiment, the data in the pre-filling stage and the decoding stage and under different configuration conditions are interpolated to obtain the energy consumption in the pre-filling stage. , Energy consumption in decoding stage .

[0102] Then, the optimal DVFS configuration can be obtained by the following formula ( , ), to set the best core frequency for the GPU and memory frequency , the formula is as follows.

[0103] ;

[0104] ;

[0105] Among them, the parameters and As an adjustable hyperparameter, it is used to weigh the trade-off between performance and energy consumption. When it is relatively large, better energy consumption optimization effect can be achieved, but it will also bring greater overhead accordingly; when it is relatively small, the overhead is reduced, and the energy consumption optimization effect is weakened accordingly. In order to implement DVFS configuration, the API (application programming interface) provided by NVML is used. These APIs can set the required voltage and frequency targets, so as to optimize the performance according to the optimal configuration parameters ( , ) to achieve the best DVFS configuration.

[0106] The energy consumption optimization strategy proposed in this embodiment can reduce the energy consumption of modern high-end GPUs by more than 20% on average, up to more than 30% through offline testing, with an average performance loss of about 10%. Take the Llama-2-7b model (a company's open source large model) running on vllm (an open source reasoning framework) as an example. Figure 3 As shown, Figure 3 When the batch size is 32 and the input length is 128, different output lengths are obtained under four different strategies ( Figure 3 The energy consumption optimization of the following (normal, pre-filling best, decoding best, and global best) is shown in the figure. The normal item represents the unoptimized situation, the pre-filling best item represents the optimization with the best DVFS configuration in the pre-filling stage, the decoding best item represents the optimization with the best DVFS configuration in the decoding stage, and the all best item represents the optimization results with the best DVFS configuration in each stage. In the online reasoning scenario, , The test results are as follows Figure 4 As shown in the figure, energy consumption decreased by 19.6% with a performance loss of 4.2%.

[0107] It is worth mentioning that in this embodiment, optimization can also be performed without stage separation. And use other information collection tools to implement the dynamic optimization process. This embodiment is mainly used for GPU energy efficiency optimization in large model reasoning scenarios. Therefore, the solution in this embodiment can be applied to other computing devices and other reasoning scenarios. As well as being extended to multi-machine environments and distributed cluster environments, it can also be applied to edge devices or mobile devices. For example, this solution is applied to a heterogeneous computing architecture (such as CPU+GPU+FPGA), and the energy efficiency data of each computing unit is collected to calculate the optimal DVFS configuration for each unit.

[0108] This embodiment achieves the following technical effects through the above technical solution:

[0109] In this embodiment, energy consumption tests are performed on data of different load sizes in an offline state of a large language model to obtain energy consumption performance under different dynamic voltage and frequency adjustment configurations; the optimal dynamic voltage and frequency adjustment configuration for each batch in the pre-filling stage and the decoding stage can be determined based on the obtained energy consumption performance under the different dynamic voltage and frequency adjustment configurations; and the GPU core frequency and memory frequency are adjusted according to the determined optimal dynamic voltage and frequency adjustment configuration to optimize the energy efficiency of large language model reasoning; this embodiment collects power and delay data of the pre-filling stage and the decoding stage of the large model reasoning, respectively, and optimizes them respectively, thereby achieving more refined optimization control and better energy efficiency optimization effect.

[0110] Exemplary Devices

[0111] Based on the above embodiments, the present invention further provides an energy efficiency optimization device based on dynamic voltage and frequency adjustment, comprising:

[0112] The energy consumption test module is used to perform energy consumption tests on data of different load sizes in the offline state of a large language model, and obtain the energy consumption performance under different dynamic voltage and frequency adjustment configurations;

[0113] A dynamic voltage and frequency adjustment configuration module is used to determine the dynamic voltage and frequency adjustment configuration of each batch in the pre-filling stage and the decoding stage according to the energy consumption performance under different dynamic voltage and frequency adjustment configurations obtained;

[0114] The energy efficiency optimization module is used to adjust the GPU core frequency and memory frequency according to the determined dynamic voltage and frequency adjustment configuration to optimize the energy efficiency of large language model inference.

[0115] This embodiment achieves the following technical effects through the above technical solution:

[0116] In this embodiment, energy consumption tests are performed on data of different load sizes in an offline state of a large language model to obtain energy consumption performance under different dynamic voltage and frequency adjustment configurations; the optimal dynamic voltage and frequency adjustment configuration for each batch in the pre-filling stage and the decoding stage can be determined based on the obtained energy consumption performance under the different dynamic voltage and frequency adjustment configurations; and the GPU core frequency and memory frequency are adjusted according to the determined optimal dynamic voltage and frequency adjustment configuration to optimize the energy efficiency of large language model reasoning; this embodiment collects power and delay data of the pre-filling stage and the decoding stage of the large model reasoning, respectively, and optimizes them respectively, thereby achieving more refined optimization control and better energy efficiency optimization effect.

[0117] Based on the above embodiment, the present invention further provides a terminal, whose principle block diagram can be as follows: Figure 5 shown.

[0118] The terminal includes: a processor, a memory, an interface, a display screen and a communication module connected through a system bus; wherein the processor of the terminal is used to provide computing and control capabilities; the memory of the terminal includes a storage medium and an internal memory; the storage medium stores an operating system and a computer program; the internal memory provides an environment for the operation of the operating system and the computer program in the storage medium; the interface is used to connect to external devices; the display screen is used to display corresponding information; and the communication module is used to communicate with a cloud server or other devices.

[0119] When the computer program is executed by a processor, the computer program is used to implement the operation of the energy efficiency optimization method based on dynamic voltage and frequency adjustment.

[0120] It can be understood by those skilled in the art that Figure 5 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal to which the scheme of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0121] In one embodiment, a terminal is provided, which includes: a processor and a memory, the memory storing an energy efficiency optimization program based on dynamic voltage and frequency adjustment, and the energy efficiency optimization program based on dynamic voltage and frequency adjustment is used to implement the operation of the above energy efficiency optimization method based on dynamic voltage and frequency adjustment when executed by the processor.

[0122] In one embodiment, a storage medium is provided, wherein the storage medium stores an energy efficiency optimization program based on dynamic voltage and frequency adjustment, and when the energy efficiency optimization program based on dynamic voltage and frequency adjustment is executed by a processor, it is used to implement the operation of the above energy efficiency optimization method based on dynamic voltage and frequency adjustment.

[0123] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and volatile memory.

[0124] In summary, the present invention provides an energy efficiency optimization method based on dynamic voltage and frequency adjustment, including: in an offline state of a large language model, performing energy consumption tests on data of different load sizes to obtain energy consumption performance under different dynamic voltage and frequency adjustment configurations; determining the optimal dynamic voltage and frequency adjustment configuration for each batch in the pre-filling stage and the decoding stage according to the obtained energy consumption performance under the different dynamic voltage and frequency adjustment configurations; adjusting the GPU core frequency and the memory frequency according to the determined optimal dynamic voltage and frequency adjustment configuration to optimize the energy efficiency of large language model reasoning; the present invention proposes an optimization scheme based on dynamic voltage and frequency adjustment (DVFS), and for the pre-filling stage and the decoding stage of large model reasoning, different dynamic voltage and frequency adjustment configurations are respectively applied to achieve energy consumption optimization.

[0125] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. An energy efficiency optimization method based on dynamic voltage and frequency adjustment, characterized in that: include: In the offline state of a large language model, energy consumption tests are performed on data of different load sizes to obtain energy consumption performance under different dynamic voltage and frequency adjustment configurations; Determine the dynamic voltage and frequency adjustment configuration of each batch in the pre-filling stage and the decoding stage according to the obtained energy consumption performance under different dynamic voltage and frequency adjustment configurations; Adjusting GPU core frequency and memory frequency based on a determined dynamic voltage frequency scaling configuration to optimize energy efficiency for large language model inference; In the offline state of the large language model, the energy consumption test is performed on data of different load sizes to obtain the energy consumption performance under different dynamic voltage and frequency adjustment configurations, including: In the offline state of a large language model, based on the power consumption test tool provided by the GPU management library, the following formula is used to test the energy consumption of data with different load sizes: ； ； ； in, Indicates load; Indicates the graphics processor core frequency; Indicates the graphics processor memory frequency; , , Respectively represent the batch size, input length, and output length of the input data; , , They represent the energy consumption, power and delay values ​​in the pre-filling phase respectively; , , They represent the energy consumption, power and delay values ​​in the decoding stage respectively.

2. The energy efficiency optimization method based on dynamic voltage and frequency adjustment according to claim 1, characterized in that: The power of the pre-filling stage and the power of the decoding stage are both average powers of the corresponding stages measured and calculated by the power consumption test tool provided by the graphics processor management library; the delay value of the pre-filling stage and the delay value of the decoding stage are both time differences between the beginning and the end of the corresponding stages.

3. The energy efficiency optimization method based on dynamic voltage and frequency adjustment according to claim 1, characterized in that: The step of determining the dynamic voltage and frequency adjustment configurations of each batch in the pre-filling stage and the decoding stage according to the obtained energy consumption performance under different dynamic voltage and frequency adjustment configurations includes: According to the calculated energy consumption of the pre-filling stage and the energy consumption of the decoding stage, the dynamic voltage and frequency adjustment configuration of different loads in the corresponding pre-filling stage and decoding stage is calculated using the following formula: ； ； in, represents the dynamic voltage and frequency adjustment configuration in the pre-filling stage; Indicates the dynamic voltage and frequency adjustment configuration at the decoding stage.

4. The energy efficiency optimization method based on dynamic voltage and frequency adjustment according to claim 1, characterized in that: The method adjusts the graphics processor core frequency and memory frequency according to the determined dynamic voltage and frequency adjustment configuration to optimize the energy efficiency of large language model reasoning, which previously includes: When the large language model is in an online state, it is dynamically determined whether a dynamic voltage and frequency adjustment optimization strategy needs to be implemented in the current stage according to the batch size, input length and corresponding stage of the online reasoning of the large language model.

5. The energy efficiency optimization method based on dynamic voltage and frequency adjustment according to claim 4, characterized in that: In the online state of the large language model, dynamically judging whether a dynamic voltage and frequency adjustment optimization strategy needs to be implemented in the current stage according to the batch size, input length and corresponding stage of the online reasoning of the large language model, including: For the inference request of the pre-filling stage, the delay value of executing the pre-filling stage once is determined according to the test result in the offline state, and the delay value is compared with the preset frequency and voltage modulation threshold. Performing a comparison, and determining, based on the comparison result, whether the dynamic voltage and frequency adjustment optimization strategy needs to be implemented at the current stage; For the inference request in the decoding stage, the duration from the last dynamic voltage and frequency adjustment optimization in the current stage is calculated based on the global timer, and the calculated duration is compared with the preset adjustment interval. A comparison is performed, and according to the comparison result, it is determined that the dynamic voltage and frequency adjustment optimization strategy needs to be implemented in the current stage.

6. The energy efficiency optimization method based on dynamic voltage and frequency adjustment according to claim 1, characterized in that: The step of adjusting the graphics processor core frequency and memory frequency according to the determined dynamic voltage frequency adjustment configuration includes: According to the determined dynamic voltage and frequency adjustment configuration, the voltage and frequency targets required by the graphics processor are set using an application program interface provided by a graphics processor management library, and the graphics processor core frequency and the memory frequency are adjusted.

7. An energy efficiency optimization device based on dynamic voltage and frequency adjustment, used to implement the energy efficiency optimization method based on dynamic voltage and frequency adjustment as claimed in any one of claims 1 to 6, characterized in that: include: The energy consumption test module is used to perform energy consumption tests on data of different load sizes in the offline state of a large language model, and obtain the energy consumption performance under different dynamic voltage and frequency adjustment configurations; A dynamic voltage and frequency adjustment configuration module is used to determine the dynamic voltage and frequency adjustment configuration of each batch in the pre-filling stage and the decoding stage according to the energy consumption performance under different dynamic voltage and frequency adjustment configurations obtained; The energy efficiency optimization module is used to adjust the graphics processor core frequency and memory frequency according to the determined dynamic voltage and frequency adjustment configuration to optimize the energy efficiency of large language model inference.

8. A terminal, characterized in that: include: A processor and a memory, wherein the memory stores an energy efficiency optimization program based on dynamic voltage and frequency adjustment, and when the energy efficiency optimization program based on dynamic voltage and frequency adjustment is executed by the processor, it is used to implement the operation of the energy efficiency optimization method based on dynamic voltage and frequency adjustment as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an energy efficiency optimization program based on dynamic voltage and frequency adjustment, and when the energy efficiency optimization program based on dynamic voltage and frequency adjustment is executed by a processor, it is used to implement the operation of the energy efficiency optimization method based on dynamic voltage and frequency adjustment as described in any one of claims 1-6.

Images