Energy efficiency optimization method based on dynamic voltage and frequency scaling

Abstract

Disclosed in the present invention is an energy efficiency optimization method based on dynamic voltage and frequency scaling (DVFS). The method comprises: in an offline state of a large language model, performing energy consumption tests on data of different load sizes, so as to obtain energy consumption performances under different DVFS configurations; on the basis of the obtained energy consumption performances under the different DVFS configurations, determining optimal DVFS configurations for each batch in a pre-filling stage and a decoding stage; and on the basis of the determined optimal DVFS configurations, scaling the core frequency and memory frequency of a graphics processing unit, so as to optimize the energy efficiency of inference performed by means of the large language model. The present invention provides a DVFS-based optimization solution, which respectively uses different DVFS configurations for a pre-filling stage and decoding stage of large model inference, thereby implementing the optimization of energy consumption.

Description

Energy efficiency optimization method based on dynamic voltage and frequency scaling TECHNICAL FIELD

[0001] The present application relates to the technical field of computer technology, and particularly relates to an energy efficiency optimization method based on dynamic voltage and frequency scaling. BACKGROUND

[0002] The inference process of large language models (LLMs) involves a large number of computations that consume a large amount of power, especially when running on specialized hardware such as GPUs (Graphics Processing Units). As the model size continues to grow, its energy consumption also shows a significant growth trend, which not only increases the running cost, but also exacerbates the impact on the surrounding environment temperature. Therefore, optimizing the energy consumption of these models not only can reduce the cost, but also helps to reduce the carbon footprint and achieve more sustainable AI technology applications. To solve this problem, dynamic voltage and frequency scaling (DVFS) has become a promising technology that can adjust devices to lower performance / power states. DVFS optimizes GPU performance and power consumption by adjusting voltage and frequency levels, providing significant energy savings with little impact on performance, allowing energy savings while maintaining the performance of large model inference.

[0003] Existing large model inference energy optimization techniques mainly focus on model compression, hardware optimization, and software and algorithm improvements. These techniques each have their advantages, but compared to the DVFS technology, which is phase-separated, there are some potential drawbacks:

[0004] 1) Model compression techniques such as pruning, knowledge distillation, and quantization can reduce model size and reduce the amount of computation during inference, thereby reducing energy consumption. However, these techniques may sacrifice the accuracy or performance of the model. Compared with DVFS technology, model compression usually needs to be done before the model is deployed, which may limit the ability of the model to dynamically adjust performance and energy consumption at runtime.

[0005] 2) Hardware optimization, such as custom accelerators or efficient data center designs, can significantly improve energy efficiency, but these optimizations often require expensive investment and may not be flexible enough to adapt to different workloads.

[0006] 3) Software-level optimization, such as efficient inference engines and algorithm improvements, can reduce energy consumption. However, these optimizations usually depend on specific algorithms or model architectures and may not be universally applicable.

[0007] Overall, while existing large model inference energy optimization techniques play an important role in reducing energy consumption, they often lack the flexibility and broad applicability of real-time, dynamic energy adjustment provided by DVFS.

[0008] Therefore, the prior art still needs to be improved. SUMMARY

[0009] The technical problem to be solved by the present application is that, in view of the defects of the prior art, the present application provides an energy efficiency optimization method based on dynamic voltage frequency adjustment to solve the problem that the existing large model inference energy consumption optimization technology lacks flexibility and wide applicability of real-time and dynamic adjustment of energy consumption.

[0010] The technical solution adopted by the present application to solve the technical problem is as follows:

[0011] In a first aspect, the present application provides an energy efficiency optimization method based on dynamic voltage frequency adjustment, comprising:

[0012] In the offline state of the large language model, energy consumption tests are performed on data of different load sizes to obtain energy consumption performance under different dynamic voltage frequency adjustment configurations;

[0013] According to the obtained energy consumption performance under different dynamic voltage frequency adjustment configurations, the dynamic voltage frequency adjustment configurations of each batch in the pre-filling stage and the decoding stage are determined;

[0014] According to the determined dynamic voltage frequency adjustment configuration, the graphics processor core frequency and the memory frequency are adjusted to optimize the energy efficiency of the large language model inference.

[0015] In one implementation, the energy consumption test on data of different load sizes in the offline state of the large language model to obtain energy consumption performance under different dynamic voltage frequency adjustment configurations comprises:

[0016] In the offline state of the large language model, based on the power consumption test tool provided by the graphics processor management library, the energy consumption of data of different load sizes is tested using the following formula: load=(bs,input-len,output-len); E prefill load,f core ,f mem =P prefill *Latency prefill ; E decoding load,f core ,f mem =P decoding *Latency decoding ;

[0017] Wherein, load represents the load; f core represents the graphics processor core frequency; f mem represents the graphics processor memory frequency;

[0018] bs, input-len, output-len represent batch size of input data, input length and output length respectively;

[0019] E prefill , P prefill , Latency prefill represent energy consumption, power and latency value of pre-filling stage respectively;

[0020] E decoding , P decoding , Latency decoding represent energy consumption, power and latency value of decoding stage respectively.

[0021] In an implementation, the power of the pre-filling stage and the power of the decoding stage are both average power of the corresponding stage calculated by power consumption test tool provided by the graphics processor management library; the latency value of the pre-filling stage and the latency value of the decoding stage are both time difference value at the beginning and end of the corresponding stage.

[0022] In an implementation, the determining of the dynamic voltage frequency adjustment configuration of each batch in the pre-filling stage and the decoding stage according to the obtained energy consumption performance under different dynamic voltage frequency adjustment configurations comprises:

[0023] According to the calculated energy consumption of the pre-filling stage and the energy consumption of the decoding stage, the dynamic voltage frequency adjustment configuration of different loads in the corresponding pre-filling stage and decoding stage is calculated by the following formula: Conf prefill = argminf core , f mem (E prefill ); Conf decoding = argminf core , f mem (E decoding );

[0024] Wherein, Conf prefill represents the dynamic voltage frequency adjustment configuration of the pre-filling stage; Conf decoding represents the dynamic voltage frequency adjustment configuration of the decoding stage.

[0025] In an implementation, the adjusting of the graphics processor core frequency and the memory frequency according to the determined dynamic voltage frequency adjustment configuration to optimize the energy efficiency of large language model inference previously comprises:

[0026] In the online state of the large language model, whether the dynamic voltage frequency adjustment optimization strategy needs to be implemented in the current stage is dynamically determined according to the batch size, input length and corresponding stage of online inference of the large language model.

[0027] In an implementation manner, the determining, in the online state of the large language model, whether the dynamic voltage and frequency adjustment optimization strategy needs to be implemented according to the batch size, input length and corresponding stage of online inference of the large language model, comprises:

[0028] For the inference request of the pre-filling stage, the delay value of executing the pre-filling stage once is determined according to the test result in the offline state, and the delay value is compared with a preset frequency and voltage adjustment threshold α, and it is determined whether the dynamic voltage and frequency adjustment optimization strategy needs to be implemented according to the comparison result;

[0029] For the inference request of the decoding stage, the time length from the last dynamic voltage and frequency adjustment optimization is calculated based on a global timer, and the calculated time length is compared with a preset adjustment interval β, and it is determined whether the dynamic voltage and frequency adjustment optimization strategy needs to be implemented according to the comparison result.

[0030] In an implementation manner, the adjusting the frequency of the graphics processor core and the memory frequency according to the determined dynamic voltage and frequency adjustment configuration comprises:

[0031] According to the determined dynamic voltage and frequency adjustment configuration, the voltage and frequency targets required by the graphics processor are set by using an application program interface provided by a graphics processor management library, and the frequency of the graphics processor core and the memory frequency are adjusted.

[0032] In a second aspect, the present application provides an energy efficiency optimization device based on dynamic voltage and frequency adjustment, comprising:

[0033] An energy consumption test module is configured to perform energy consumption tests on data of different load sizes in an offline state of a large language model, and obtain energy consumption performances under different dynamic voltage and frequency adjustment configurations;

[0034] A dynamic voltage and frequency adjustment configuration module is configured to determine dynamic voltage and frequency adjustment configurations of each batch in the pre-filling stage and the decoding stage according to the obtained energy consumption performances under different dynamic voltage and frequency adjustment configurations;

[0035] An energy efficiency optimization module is configured to adjust the frequency of the graphics processor core and the memory frequency according to the determined dynamic voltage and frequency adjustment configuration, so as to optimize the energy efficiency of large language model inference.

[0036] In a third aspect, the present application provides a terminal, comprising a processor and a memory, the memory storing a dynamic voltage frequency adjustment based energy efficiency optimization program, the dynamic voltage frequency adjustment based energy efficiency optimization program being used to implement the operations of the dynamic voltage frequency adjustment based energy efficiency optimization method according to the first aspect when executed by the processor.

[0037] In a fourth aspect, the present application further provides a medium, which is a computer readable storage medium, the medium storing a dynamic voltage frequency adjustment based energy efficiency optimization program, the dynamic voltage frequency adjustment based energy efficiency optimization program being used to implement the operations of the dynamic voltage frequency adjustment based energy efficiency optimization method according to the first aspect when executed by a processor.

[0038] The present application has the following effects by adopting the above technical solutions:

[0039] The present application performs energy consumption tests on data of different load sizes in the offline state of the large language model, and obtains energy consumption performances under different dynamic voltage frequency adjustment configurations; the energy consumption performances under different dynamic voltage frequency adjustment configurations are used to determine the optimal dynamic voltage frequency adjustment configurations of each batch in the pre-padding stage and the decoding stage; and the graphic processor core frequency and the memory frequency are adjusted according to the determined optimal dynamic voltage frequency adjustment configurations, so as to optimize the energy efficiency of the large language model inference; the present application respectively collects power and delay data of the pre-padding stage and the decoding stage of the large model inference, and respectively optimizes the two stages, so as to achieve more fine optimization control and better energy efficiency optimization effect. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from the structures shown in the drawings without creative labor.

[0041] Fig. 1 is a flowchart of the dynamic voltage frequency adjustment based energy efficiency optimization method according to the present application.

[0042] Fig. 2 is a schematic diagram of the dynamic voltage frequency adjustment based energy efficiency optimization according to the present application.

[0043] Fig. 3 is a schematic diagram of part of the energy consumption of the large language model in the offline test according to the present application.

[0044] Fig. 4 is a schematic diagram of the online inference energy consumption optimization effect of the large language model according to the present application.

[0045] Fig. 5 is a functional schematic diagram of a terminal in an implementation of the present application.

[0046] The purposes, technical solutions and advantages of the present application will be further described in detail with reference to the accompanying drawings and embodiments. DETAILED DESCRIPTION

[0047] To make the purposes, technical solutions and advantages of the present application clearer and more explicit, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0048] Exemplary method

[0049] There are some potential drawbacks in existing large model inference energy optimization techniques: first, using model compression techniques (such as pruning, knowledge distillation and quantization) can reduce model size, reduce computational load in inference process, and thus reduce energy consumption. However, these techniques may sacrifice the accuracy or performance of the model. Compared with DVFS (dynamic voltage and frequency scaling) technology, model compression usually needs to be done before model deployment, which may limit the ability of the model to dynamically adjust performance and energy consumption at runtime. Second, using hardware optimization techniques (such as custom accelerators or efficient data center design), although it can significantly improve energy efficiency, these optimizations often require expensive investment and may not be flexible enough to adapt to different workloads. Finally, using software-level optimization techniques (such as efficient inference engines and algorithm improvements), can reduce energy consumption. However, these optimizations usually depend on specific algorithms or model architectures, and may not be universally applicable.

[0050] Overall, although existing large model inference energy optimization techniques play an important role in reducing energy consumption, they usually lack the flexibility and wide applicability of real-time and dynamic energy adjustment provided by DVFS.

[0051] To solve the above technical problems, the embodiment of the present application provides an energy efficiency optimization method based on dynamic voltage frequency adjustment, which mainly tests the energy consumption of data with different load sizes in the offline state of a large language model to obtain the energy consumption performance under different dynamic voltage frequency adjustment configurations; determines the optimal dynamic voltage frequency adjustment configuration of each batch in the pre-padding stage and the decoding stage according to the obtained energy consumption performance under different dynamic voltage frequency adjustment configurations; and adjusts the GPU (graphics processing unit) core frequency and memory frequency according to the determined optimal dynamic voltage frequency adjustment configuration to optimize the energy efficiency of large language model inference; therefore, the embodiment of the present application can collect power and delay data of the pre-padding stage and the decoding stage of large model inference respectively, and optimize them respectively, realizing more fine optimization control and better energy efficiency optimization effect.

[0052] As shown in FIG. 1, the embodiment of the present application provides an energy efficiency optimization method based on dynamic voltage frequency adjustment, comprising the following steps:

[0053] Step S100, in the offline state of a large language model, test the energy consumption of data with different load sizes to obtain the energy consumption performance under different dynamic voltage frequency adjustment configurations.

[0054] In the embodiment, an optimization scheme based on dynamic voltage frequency adjustment (DVFS) is proposed, which applies different dynamic voltage frequency adjustment configurations to realize energy consumption optimization for the pre-padding stage and the decoding stage of large language model inference.

[0055] As shown in FIG. 2, in the embodiment, first, in the offline state of a large language model, test data with different load sizes (i.e. batch size, input length, and output length of a large language model) to measure the energy consumption performance under different DVFS configurations (dynamic voltage frequency adjustment configuration), so as to determine the optimal DVFS configuration of each batch in the pre-padding stage and the decoding stage according to the energy consumption performance; then, in the online inference scenario of a large language model, dynamically determine whether to implement dynamic voltage frequency adjustment optimization according to the load (such as batch size and input length) and different stages. When the pre-padding stage delay is greater than a threshold α or the decoding stage interval is greater than β, the actual load energy consumption data is obtained by quadratic interpolation according to the offline measured energy consumption data, and then the optimal DVFS configuration is solved, the GPU core frequency and memory frequency are adjusted according to the configuration, and energy consumption optimization is realized.

[0056] Specifically, in an implementation manner of the embodiment, step S100 comprises the following steps:

[0057] Step S101, in the offline state of the large language model, energy consumption tests are performed on data of different load sizes based on a power consumption test tool provided by an NVML library (i.e., a graphics processor management library provided by a certain company).

[0058] In this embodiment, first, data of different load sizes are tested in an offline state, and energy consumption performance under different DVFS configurations (f core , f_mem) is measured, as shown in the following formula: load = (bs, input-len, output-len); E prefill load, f core load, f mem = P prefill *Latency prefill ; E decoding load, f core load, f mem = P decoding *Latency decoding ;

[0059] wherein load represents the load; f core represents the GPU kernel frequency; f mem represents the GPU memory frequency; bs, input-len, and output-len respectively represent the batch size of input data, the input length, and the output length; E prefill , P prefill , and Latency prefill respectively represent the energy consumption, power, and latency value in the pre-filling stage; E decoding , P decoding , and Latency decoding respectively represent the energy consumption, power, and latency value in the decoding stage.

[0060] In this embodiment, for the batch size, input length, and output length of the input data of the large language model inference process, the batch size refers to the number of data samples processed simultaneously in a single inference process; the input length refers to the size or length of a single input data accepted by the model; and the output length usually refers to the size or length of the output data generated by the model. In the process of energy consumption testing on data of different load sizes, the power P prefill in the pre-filling stage and the power P decoding, which are average power of corresponding stage calculated by power test tool provided by NVML library; the specific calculation process is: using power test tool provided by NVML library, power is measured every certain time interval t, and finally average power of corresponding stage can be obtained according to recorded log. And the delay value of pre-filling stage and the delay value of decoding stage are time difference values at the beginning and end of corresponding stage, that is, only need to count time at the beginning and end of each stage respectively, and then the difference can be obtained.

[0061] In the embodiment, through the above test process of data with different load sizes in offline state, power consumption data E prefill corresponding to pre-filling stage of data with different load sizes can be obtained. decoding According to power consumption data corresponding to different stages of different loads, the optimal DVFS configuration of each batch in pre-filling stage and decoding stage can be calculated.

[0062] As shown in FIG. 1, in an implementation manner of the embodiment of the application, the energy efficiency optimization method based on dynamic voltage frequency adjustment further includes the following steps:

[0063] In step S200, dynamic voltage frequency adjustment configuration of each batch in pre-filling stage and decoding stage is determined according to obtained energy consumption performance under different dynamic voltage frequency adjustment configurations.

[0064] In the embodiment, according to power consumption data E prefill corresponding to pre-filling stage and power consumption data E decoding corresponding to decoding stage, the optimal DVFS configuration of different loads in pre-filling stage and decoding stage can be obtained.

[0065] Specifically, in an implementation manner of the embodiment, step S200 includes the following steps:

[0066] In step S201, dynamic voltage frequency adjustment configuration of different loads in corresponding pre-filling stage and decoding stage is calculated according to calculated energy consumption of pre-filling stage and energy consumption of decoding stage.

[0067] In the embodiment, the calculation manner of the optimal DVFS configuration of different loads in pre-filling stage and decoding stage is: Conf prefill = argmin f core , f mem (E prefill ); Conf decoding = argmin f core , f mem (Edecoding

[0068] wherein, Conf prefill represents the optimal DVFS configuration of the pre-padding stage; Conf decoding represents the optimal DVFS configuration of the decoding stage.

[0069] In this embodiment, the optimal DVFS configurations corresponding to the two stages (pre-padding stage and decoding stage) of the large language model in the offline state are calculated according to the above manner; according to the optimal DVFS configurations in the offline state, the energy consumption of the GPU can be adjusted in real time and dynamically in the online state of the large language model.

[0070] As shown in FIG. 1, in an implementation manner of the embodiment of the present application, the energy efficiency optimization method based on dynamic voltage frequency adjustment further includes the following steps:

[0071] Step S300, adjusting the GPU core frequency and memory frequency according to the determined dynamic voltage frequency adjustment configuration, so as to optimize the energy efficiency of the large language model inference.

[0072] It should be noted that in this embodiment, in the online inference scenario of the large language model, before dynamically optimizing the energy consumption of the GPU, it is also necessary to dynamically determine whether the current inference stage needs to implement the dynamic voltage frequency adjustment optimization process according to the current load (i.e. batch size and input length) and the corresponding stage of the online inference.

[0073] Specifically, in an implementation manner of this embodiment, step S300 includes the following steps before it:

[0074] Step S301a, in the online state of the large language model, dynamically determining whether the current stage needs to implement the dynamic voltage frequency adjustment optimization strategy according to the batch size, input length and corresponding stage of the online inference of the large language model.

[0075] ​Specifically, in an implementation manner of the embodiment, in the online state of the large language model, whether the DVFS optimization strategy (i.e., a dynamic voltage and frequency adjustment optimization strategy) needs to be implemented is dynamically determined according to a batch size, an input length and a corresponding stage of online inference of the large language model, including: for an inference request of the pre-padding stage, a delay value of executing the pre-padding stage once is determined according to a test result in the offline state, the delay value is compared with a preset threshold α of frequency and voltage adjustment, and whether the DVFS optimization strategy needs to be implemented is determined according to a comparison result; for an inference request of the decoding stage, a time length from last time of performing the DVFS optimization is calculated based on a global timer, the calculated time length is compared with a preset adjustment interval β, and whether the DVFS optimization strategy needs to be implemented is determined according to a comparison result.

[0076] As an example, in the embodiment, in the process of dynamically determining whether the DVFS optimization needs to be implemented, the basis for the determination is that, for a request to be pre-filled, a delay value of executing the pre-padding stage once is determined according to a test in the offline state, and then the delay value is compared with a preset threshold α of frequency and voltage adjustment corresponding to the pre-padding stage. If the delay value exceeds the threshold, the DVFS optimization needs to be performed. For a decoding stage request, a preset adjustment interval β is set, and there is a global timer. Whether the time from last time of performing the DVFS optimization reaches the interval is determined through the global timer. When the interval is reached, the DVFS optimization needs to be performed.

[0077] In the application, the values of α and β are experience values determined according to the degree of preference of the system for performance and energy efficiency. The smaller α is, the more the performance of the pre-padding stage is emphasized, and vice versa. The smaller β is, the more the performance of the decoding stage is emphasized, and vice versa. The setting of α and β is mainly based on actual application requirements. By setting the values of α and β, the trade-off between performance and energy consumption is realized.

[0078] Regarding the value range of α and β, the following is an example:

[0079] The value range of α is:

[0080] When the system has a higher requirement for performance, α should be set to a larger value. For example, α can be taken to be between 0.5 and 1.0, so as to more emphasize the performance of the pre-padding stage.

[0081] If the system pays more attention to energy efficiency, i.e., wants to save more energy consumption in the pre-padding stage, α can be set to a smaller value, which can be in the range of 0.1-0.5.

[0082] The value range of β is:

[0083] When the performance of the decoding stage is more critical, the value of β should be larger. For example, β can be taken between 0.5-1.0 to ensure high performance of the decoding stage.

[0084] Conversely, if the decoding stage is more inclined to save energy, the value of β should be set to be smaller, which can be in the range of 0.1-0.5.

[0085] Specifically, in an implementation manner of the embodiment, step S300 includes the following steps:

[0086] Step S301, according to the determined dynamic voltage frequency adjustment configuration, set the voltage and frequency targets required by the GPU by using the API provided by the NVML library, adjust the GPU core frequency and the memory frequency.

[0087] In the embodiment, the specific optimization process is as follows: for the batch size and input length of the current scheduling request, the embodiment can find the bs1 (batch size 1), input-len1 (input length 1), bs2 (batch size 2) and input-len2 (input length 2) corresponding to the offline state measured near the load. Here, the selection of the output length is not limited, because under the same other conditions, different output lengths show the same DVFS characteristics. Therefore, the embodiment can obtain the energy consumption corresponding to the current load through quadratic interpolation:

[0088] In the embodiment, the data in the pre-padding stage and the decoding stage under different configuration conditions are respectively interpolated, and thus the pre-padding stage energy consumption E prefill load,f core ,f mem , the decoding stage energy consumption E decoding load,f core ,f mem .

[0089] Then, the optimal DVFS configuration (Conf prefill , Conf decoding ) can be obtained by the following formula to set the optimal core frequency f core and the optimal memory frequency f mem for the GPU. The formula is as follows. Conf prefill =argminf core ,f mem (E prefill ); Conf decoding =argminf core ,f mem (E decoding );

[0090] Wherein, the parameters a and b are adjustable hyperparameters, used to balance the trade-off between performance and energy consumption. When relatively large, better energy optimization results can be obtained, but it will also bring more overhead; when relatively small, the overhead is reduced, and the energy optimization effect is also weakened. In order to realize the DVFS configuration, the API (application programming interface) provided by NVML is used. These APIs can set the required voltage and frequency targets, so as to realize the best DVFS configuration according to the optimal configuration parameters (Conf prefill 、Conf decoding ).

[0091] The energy optimization strategy proposed in this embodiment can reduce the energy consumption of a modern high-end GPU by more than 20% on average, and by more than 30% at most, with an average performance loss of about 10%. Taking the Llama-2-7b model (a company's open source large model) running on vllm (an open source inference framework) as an example, as shown in FIG. 3, the batch size is 32, the input length is 128, and the energy optimization of different output lengths under four different strategies (normal, pre-padding best, decoding best, and global best shown in FIG. 3). Among them, the normal item represents the unoptimized case, the pre-padding best item represents optimization with the best DVFS configuration in the pre-padding stage, the decoding best item represents optimization with the best DVFS configuration in the decoding stage, and the overall best represents the optimization result of each stage with the best DVFS configuration. In the online inference scenario, the test results of a = 0.1s and b = 1s are shown in FIG. 4. In the case of a performance loss of 4.2%, the energy consumption is reduced by 19.6%.

[0092] It is worth mentioning that in this embodiment, optimization can also be performed without stage separation. And use other information collection tools, and then realize the process of dynamic optimization. The main application in this embodiment is the energy efficiency optimization of GPU in large model inference scenarios, so the scheme in this embodiment can be applied to other computing devices and other inference scenarios. And extended to multi-machine environment and distributed cluster environment, it can also be applied to edge devices or mobile devices. For example, apply this scheme to a heterogeneous computing architecture (such as CPU + GPU + FPGA), and calculate the best DVFS configuration for each unit by collecting the energy efficiency data of each computing unit.

[0093] The above technical solutions achieve the following technical effects:

[0094] The embodiment tests the energy consumption of data with different load sizes in the offline state of the large language model, obtains the energy consumption performance under different dynamic voltage frequency adjustment configurations, determines the optimal dynamic voltage frequency adjustment configuration of each batch in the pre-padding stage and the decoding stage according to the obtained energy consumption performance under different dynamic voltage frequency adjustment configurations, and adjusts the GPU core frequency and the memory frequency according to the determined optimal dynamic voltage frequency adjustment configuration to optimize the energy efficiency of the large language model inference.

[0095] Exemplary device

[0096] Based on the above embodiment, the application further provides an energy efficiency optimization device based on dynamic voltage frequency adjustment, comprising:

[0097] The energy consumption testing module is configured to test the energy consumption of data with different load sizes in the offline state of the large language model, and obtain the energy consumption performance under different dynamic voltage frequency adjustment configurations.

[0098] The dynamic voltage frequency adjustment configuration module is configured to determine the dynamic voltage frequency adjustment configuration of each batch in the pre-padding stage and the decoding stage according to the obtained energy consumption performance under different dynamic voltage frequency adjustment configurations.

[0099] The energy efficiency optimization module is configured to adjust the GPU core frequency and the memory frequency according to the determined dynamic voltage frequency adjustment configuration to optimize the energy efficiency of the large language model inference.

[0100] The embodiment achieves the following technical effects through the above technical solution:

[0101] The embodiment tests the energy consumption of data with different load sizes in the offline state of the large language model, obtains the energy consumption performance under different dynamic voltage frequency adjustment configurations, determines the optimal dynamic voltage frequency adjustment configuration of each batch in the pre-padding stage and the decoding stage according to the obtained energy consumption performance under different dynamic voltage frequency adjustment configurations, and adjusts the GPU core frequency and the memory frequency according to the determined optimal dynamic voltage frequency adjustment configuration to optimize the energy efficiency of the large language model inference.

[0102] Based on the above embodiment, the application further provides a terminal, and the principle block diagram thereof can be as shown in FIG. 5.

[0103] The terminal comprises a processor, a memory, an interface, a display screen and a communication module connected through a system bus; the processor of the terminal is configured to provide computing and control capabilities; the memory of the terminal comprises a storage medium and an internal memory; the storage medium stores an operating system and a computer program; the internal memory provides an environment for the operating system and the computer program in the storage medium to run; the interface is configured to connect external devices; the display screen is configured to display corresponding information; and the communication module is configured to communicate with a cloud server or other devices.

[0104] The computer program is configured to implement the operations of the energy efficiency optimization method based on dynamic voltage and frequency adjustment when executed by the processor.

[0105] Those skilled in the art can understand that the principle block diagram shown in FIG. 5 is only a block diagram of part of the structure related to the present application scheme, and does not constitute a limitation on the terminal to which the present application scheme is applied; specifically, the terminal can comprise more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0106] In one embodiment, a terminal is provided, comprising a processor and a memory, wherein the memory stores an energy efficiency optimization program based on dynamic voltage and frequency adjustment, and the energy efficiency optimization program based on dynamic voltage and frequency adjustment is configured to implement the operations of the energy efficiency optimization method based on dynamic voltage and frequency adjustment as described above when executed by the processor.

[0107] In one embodiment, a storage medium is provided, wherein the storage medium stores an energy efficiency optimization program based on dynamic voltage and frequency adjustment, and the energy efficiency optimization program based on dynamic voltage and frequency adjustment is configured to implement the operations of the energy efficiency optimization method based on dynamic voltage and frequency adjustment as described above when executed by the processor.

[0108] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile storage medium, and when executed, can include the processes of the above-mentioned embodiments. Any reference to the memory, storage, database or other medium used in each embodiment of the present application can include non-volatile and volatile memory.

[0109] In summary, the application provides an energy efficiency optimization method based on dynamic voltage frequency adjustment, comprising: in the offline state of the large language model, energy consumption tests are performed on data of different load sizes to obtain energy consumption performance under different dynamic voltage frequency adjustment configurations; according to the obtained energy consumption performance under different dynamic voltage frequency adjustment configurations, the optimal dynamic voltage frequency adjustment configuration of each batch in the pre-filling stage and the decoding stage is determined; the GPU core frequency and the memory frequency are adjusted according to the determined optimal dynamic voltage frequency adjustment configuration to optimize the energy efficiency of the large language model inference; the application proposes an optimization scheme based on dynamic voltage frequency adjustment (DVFS), and different dynamic voltage frequency adjustment configurations are applied to the pre-filling stage and the decoding stage of the large model inference to realize energy consumption optimization.

[0110] It should be understood that the application is not limited to the above examples, and those skilled in the art can make improvements or changes according to the above description, and all these improvements and changes shall belong to the protection scope of the appended claims of the application.

Claims

1. A method for energy efficiency optimization based on dynamic voltage and frequency scaling, comprising: The method comprises the following steps: In the offline state of the large language model, energy consumption tests are performed on data of different load sizes to obtain energy consumption performances under different dynamic voltage and frequency adjustment configurations; According to the obtained energy consumption performances under different dynamic voltage and frequency adjustment configurations, dynamic voltage and frequency adjustment configurations for each batch in the pre-filling stage and the decoding stage are determined; According to the determined dynamic voltage and frequency adjustment configurations, the frequency of the graphics processor core and the frequency of the memory are adjusted to optimize the energy efficiency of the large language model inference.

2. The energy efficiency optimization method based on dynamic voltage and frequency scaling according to claim 1, wherein, The energy consumption tests on data of different load sizes in the offline state of the large language model to obtain energy consumption performances under different dynamic voltage and frequency adjustment configurations comprise: In the offline state of the large language model, based on the power consumption test tool provided by the graphics processor management library, the following formula is used to perform energy consumption tests on data of different load sizes: load=(bs,input-len,output-len); E prefill (load,f core ,f mem )=P prefill *Latency prefill ; E decoding (load,f core ,f mem )=P decoding *Latency decoding ; wherein load represents a load; f core represents a graphics processor core frequency; f mem represents a graphics processor memory frequency; bs, input-len, and output-len represent the batch size of input data, the input length, and the output length, respectively; E prefill 、P prefill 、Latency prefill respectively represent the energy consumption, power and latency values of the pre-filling phase; E decoding 、P decoding 、Latency decoding represent the energy, power and latency values of the decoding stage, respectively.

3. The energy efficiency optimization method based on dynamic voltage and frequency scaling according to claim 2, wherein, The power of the pre-filling stage and the power of the decoding stage are both the average power of the corresponding stage calculated by the power consumption test tool provided by the graphics processor management library; the delay value of the pre-filling stage and the delay value of the decoding stage are both the time difference between the start and end of the corresponding stage.

4. The energy efficiency optimization method based on dynamic voltage and frequency scaling according to claim 1, wherein, According to the calculated energy consumption of the pre-filling stage and the energy consumption of the decoding stage, the following formula is used to calculate the dynamic voltage and frequency adjustment configurations for different loads in the corresponding pre-filling stage and decoding stage: According to the determined dynamic voltage and frequency adjustment configurations, the frequency of the graphics processor core and the frequency of the memory are adjusted to optimize the energy efficiency of the large language model inference, which previously comprises: Conf prefill = argminf core ,f mem (E prefill ); Conf decoding = argminf core ,f mem (E decoding ); wherein Conf prefill represents a dynamic voltage frequency scaling configuration for the pre-charge phase; Conf decoding represents a dynamic voltage frequency scaling configuration for the decode phase.

5. The energy efficiency optimization method based on dynamic voltage and frequency scaling according to claim 1, wherein, In the online state of the large language model, according to the batch size, input length, and corresponding stage of the online inference of the large language model, it is dynamically determined whether the dynamic voltage and frequency adjustment optimization strategy needs to be implemented in the current stage. In the online state of the large language model, according to the batch size, input length, and corresponding stage of the online inference of the large language model, it is dynamically determined whether the dynamic voltage and frequency adjustment optimization strategy needs to be implemented in the current stage, which comprises:

6. The energy efficiency optimization method based on dynamic voltage and frequency scaling according to claim 5, wherein, For the inference request of the pre-filling stage, the delay value of executing the pre-filling stage once is determined according to the test result in the offline state, and the delay value is compared with a preset frequency and voltage adjustment threshold α, and according to the comparison result, it is determined whether the dynamic voltage and frequency adjustment optimization strategy needs to be implemented in the current stage; For the inference request of the decoding stage, the time length from the last dynamic voltage and frequency adjustment optimization is calculated based on a global timer, and the calculated time length is compared with a preset adjustment interval β, and according to the comparison result, it is determined whether the dynamic voltage and frequency adjustment optimization strategy needs to be implemented in the current stage. ​ 7. The dynamic voltage and frequency scaling based energy efficiency optimization method of claim 1, wherein, The adjusting the GPU core frequency and the memory frequency according to the determined dynamic voltage and frequency adjustment configuration comprises: According to the determined dynamic voltage and frequency adjustment configuration, the voltage and frequency targets required by the GPU are set by using an application programming interface provided by a GPU management library, and the GPU core frequency and the memory frequency are adjusted.

8. A dynamic voltage and frequency scaling based energy efficiency optimization apparatus, comprising: Comprise: The energy consumption test module is used for testing the energy consumption of data with different load sizes in the offline state of the large language model, and obtaining the energy consumption performance under different dynamic voltage and frequency adjustment configurations; The dynamic voltage and frequency adjustment configuration module is used for determining the dynamic voltage and frequency adjustment configuration of each batch in the pre-padding stage and the decoding stage according to the obtained energy consumption performance under different dynamic voltage and frequency adjustment configurations; The energy efficiency optimization module is used for adjusting the GPU core frequency and the memory frequency according to the determined dynamic voltage and frequency adjustment configuration, so as to optimize the energy efficiency of the large language model inference.

9. A terminal, characterized by comprising: Comprise: The processor and the memory, the memory stores the energy efficiency optimization program based on dynamic voltage and frequency adjustment, the energy efficiency optimization program based on dynamic voltage and frequency adjustment is used to realize the operation of the energy efficiency optimization method based on dynamic voltage and frequency adjustment as claimed in any one of claims 1-7 when the processor is executed.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores the energy efficiency optimization program based on dynamic voltage and frequency adjustment, the energy efficiency optimization program based on dynamic voltage and frequency adjustment is used to realize the operation of the energy efficiency optimization method based on dynamic voltage and frequency adjustment as claimed in any one of claims 1-7 when the processor is executed.

Images