Resource optimization and carbon emission reduction method for large language model, and device

Abstract

Provided in the embodiments of the present application are a resource optimization and carbon emission reduction method for a large language model, and a device. For a pre-training process of a large language model, on the basis of a carbon emission intensity which dynamically changes within a selected time interval, a pre-training start time node and a pre-training end time node within the time interval and GPU resource amount regulation parameter values during pre-training are determined for the large language model, such that the corresponding estimated total amount of carbon emissions for pre-training of the large language model within the time interval reaches a specified requirement. In this way, the pre-training start time node, the pre-training end time node, and the GPU resource amount regulation parameter values at various time nodes during pre-training can be rationally regulated within the selected time interval on the basis of the carbon emission intensity which dynamically changes, so that the amount of GPU resources for the pre-training of the large language model is dynamically regulated, and a pre-training task is thus rationally divided into relevant time nodes within the selected time interval, thereby effectively reducing the amount of carbon emissions generated during the pre-training of the large language model.

Description

A resource optimization and carbon emission reduction method and device for large language models Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for resource optimization and carbon emission reduction for a large language model. Background Art

[0002] With the release of various generative AI approaches, large language models (LLMs) have become a research hotspot and are beginning to be widely applied in various fields. In recent years, the parameter size of large language models has continued to increase, significantly improving their performance. However, larger models require greater computing resources, which in turn means greater energy demand and increased carbon emissions.

[0003] Currently, most existing research solutions to optimize the energy consumption and carbon emissions of large language models reduce computing energy consumption by limiting chip power consumption, optimizing the neural network model structure, or selecting processors with lower energy consumption and higher efficiency. However, most existing research solutions have not conducted in-depth research on low-carbon optimization and control strategies when pre-training large language models on GPUs.

[0004] Therefore, there is an urgent need to provide better carbon emission reduction solutions for large language models.

[0005] Summary of the Invention

[0006] Multiple aspects of the present application provide a resource optimization and carbon emission reduction method and device for large language models, which are used to better reduce the carbon emissions generated by pre-training of large language models by optimizing and regulating GPU resources.

[0007] The present application provides a resource optimization and carbon emission reduction method for a large language model, including:

[0008] Select a time interval for pre-training the large language model to be processed;

[0009] Based on the dynamically changing carbon emission intensity within the time interval, determining the pre-training start and end time nodes within the time interval and the GPU resource control parameter values ​​at each time node during the pre-training for the large language model, so that the estimated total pre-training carbon emissions corresponding to the large language model within the time interval meet the specified requirements;

[0010] According to the pre-training start and end time nodes and the respective GPU resource control parameter values, the GPU resource amount used for pre-training the large language model is dynamically controlled within the time interval.

[0011] An embodiment of the present application further provides a computing device, including a memory and a processor;

[0012] The memory is configured to store one or more computer instructions;

[0013] The processor is coupled to the memory and configured to execute the one or more computer instructions to perform the aforementioned resource optimization and carbon emission reduction method for a large language model.

[0014] An embodiment of the present application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the aforementioned resource optimization and carbon emission reduction method for a large language model.

[0015] In an embodiment of the present application, for the pre-training process of a large language model, based on the dynamically changing carbon emission intensity within a selected time interval, the pre-training start and end time nodes within the time interval and the GPU resource quantity control parameter values ​​at each time node during the pre-training period are determined for the large language model, so that the total estimated pre-training carbon emissions corresponding to the large language model within the time interval meet the specified requirements; based on this, the GPU resource quantity used for pre-training of the large language model can be dynamically controlled within the time interval according to the pre-training start and end time nodes and each GPU resource quantity control parameter value determined above. In this way, the pre-training start and end time nodes and the GPU resource quantity control parameter values ​​at each time node during the pre-training period can be reasonably controlled within the selected time interval based on the dynamically changing carbon emission intensity, thereby dynamically controlling the GPU resource quantity used for pre-training of the large language model, and then reasonably dividing the pre-training tasks into relevant time nodes within the selected time interval, which can effectively reduce the carbon emissions generated by the pre-training of the large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] FIG1 is a flow chart of a method for resource optimization and carbon emission reduction for a large language model provided by an exemplary embodiment of the present application;

[0018] FIG2 is a logical diagram of a resource optimization and carbon emission reduction method for a large language model provided by an exemplary embodiment of the present application;

[0019] FIG3 is a schematic structural diagram of a computing device provided by another exemplary embodiment of the present application. DETAILED DESCRIPTION

[0020] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0021] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0022] Figure 1 is a flow chart of a method for resource optimization and carbon emission reduction for a large language model provided by an exemplary embodiment of the present application. The method can be executed by a data processing device, which can be implemented as a combination of software and / or hardware, and the data processing device can be integrated into a computing device.

[0023] The resource optimization and carbon emission reduction method for a large language model provided in this embodiment can be applied to various scenarios where carbon emission reduction of a large language model is required, and this embodiment does not limit the application scenario. Generally, a large language model needs to be deployed in a computing system comprising many servers. In actual applications, the computing system here can be a cloud data center. In this case, the computing device used to execute the resource optimization and carbon emission reduction method for a large language model in this embodiment can be a resource scheduling device deployed in the data center, and the resource optimization and carbon emission reduction method for a large language model provided in this embodiment can be a newly added functional module in the resource scheduling device deployed in the data center. Of course, the computing system in this embodiment can also be other types of systems, such as an enterprise computer room. The computing device in this embodiment can be a device with global GPU resource management authority in the computing system. In addition, the computing device in this embodiment can be a single server, or of course it can be a server cluster, etc. This embodiment does not limit the physical implementation form of the computing device.

[0024] During the research process, the inventors discovered that the carbon emissions of large language models are mainly generated in three stages: one is the carbon emissions in the equipment manufacturing stage, the second is the carbon emissions in the model pre-training stage, and the third is the carbon emissions in the model application and reasoning stage. Among them, the carbon emissions in the equipment manufacturing stage are contained in hardware equipment such as servers, and this part of carbon emissions has been determined in the production stage. The energy consumption in the model application and reasoning stages is closely related to the scale and access needs of users. Since users' questions and needs need to be responded to in a timely manner, this part of carbon emissions is more difficult to optimize and control. The pre-training stage of the large language model has a greater demand for computing resources, and usually has a relatively long training cycle (ranging from a few weeks to a few months), and is more concentrated in time than the application stage. Therefore, this embodiment proposes to explore how to reduce the carbon emissions generated by the pre-training of the large language model during the pre-training stage of the large language model.

[0025] Figure 2 is a logical diagram of a resource optimization and carbon reduction method for a large language model, provided by an exemplary embodiment of the present application. Referring to Figure 2 , in this embodiment, GPU resources can be used to carry out pre-training tasks for a large language model. GPU resources can be installed on a server. In this embodiment, a computing device can execute the method provided in this embodiment to centrally manage all GPU resources in the system.

[0026] Referring to FIG1 , the resource optimization and carbon emission reduction method for a large language model provided in this embodiment may include:

[0027] Step 100: Select a time interval for pre-training the large language model to be processed;

[0028] Step 101: Based on the dynamically changing carbon emission intensity within the time interval, determine the pre-training start and end time nodes within the time interval and the GPU resource control parameter values ​​at each time node during the pre-training for the large language model, so that the total estimated carbon emission of the pre-training corresponding to the large language model within the time interval meets the specified requirements;

[0029] Step 102: Dynamically adjust the amount of GPU resources used for pre-training the large language model within a time interval according to the pre-training start and end time nodes and the values ​​of the various GPU resource control parameters.

[0030] In actual applications, a delay threshold is usually specified for the pre-training task of a large language model. In step 100, the time interval selected for pre-training of the large language model should be greater than the preset delay threshold. Of course, more filtering conditions can be added to select a suitable time interval for the large language model to be processed. The filtering conditions are not exhaustive here.

[0031] On this basis, resource optimization and carbon emission reduction plans for large language models can be determined within the selected time interval.

[0032] Continuing with reference to FIG1 , in step 101, the pre-training start and end time nodes within the time interval and the GPU resource control parameter values ​​at each time node during the pre-training period can be determined for the large language model. It should be understood that step 101 is a parameter determination step completed before the GPU resource control operation is implemented. That is, before executing step 102, the parameters used in the implementation of the GPU resource control operation are first determined based on step 101, so that the GPU resource control operation in step 102 can be implemented in step 102 according to the predetermined parameters.

[0033] In this embodiment, the unit corresponding to the time node defined in the time interval is not limited. In this embodiment, the unit corresponding to the time node can be day, week, hour, etc., which are not exhaustively listed here.

[0034] In this embodiment, it is proposed that the GPU resource quantity control parameter values ​​at the pre-training start and end time nodes within the selected time interval and at each time node during the pre-training period can be determined for the large language model based on the dynamically changing carbon emission intensity within the selected time interval. Moreover, the GPU resource quantity control parameter values ​​at the pre-training start and end time nodes and at each time node during the pre-training period determined for the large language model must ensure that the estimated total amount of pre-training carbon emissions corresponding to the large language model within the selected time interval meets the specified requirements. The estimated total amount of pre-training carbon emissions refers to the estimated total amount of carbon emissions required to complete the pre-training of the large language model. The specified requirements may be that the estimated total amount of pre-training carbon emissions is the lowest, etc.

[0035] Here, carbon emission intensity refers to the amount of carbon emissions required to produce one unit of energy. For example, in this embodiment, the GPU resources consume electricity, so the carbon emission intensity here can refer to the amount of carbon emissions required to produce one kilowatt-hour of electricity. In practical applications, many types of energy are used to produce energy, including but not limited to light, wind, water, and coal. Different types of energy correspond to different carbon emission intensities, and the energy ratios typically change dynamically. This results in the carbon emission intensity corresponding to the electricity used by the GPU resources at different time points also changing dynamically. In this embodiment, the carbon emission intensity at each time point within a selected time interval can be determined based on the energy usage of the computing system connected to the large language model at different time points. Optionally, a weighted average of the carbon emission intensities of each type of energy used at a single time point can be calculated as the carbon emission intensity at the corresponding time point in step 101. Furthermore, the carbon emission intensities at each time point used in step 101 in this embodiment are predicted values. This embodiment is not limited to a specific prediction scheme. In this embodiment, any scheme that can predict carbon emission intensities at future time points, whether currently available or in the future, can be used.

[0036] The GPU resource quantity control parameter value in this embodiment is used to control the GPU resource quantity used for pre-training of the large language model at the corresponding time node. That is, it can be used as a basis for regulating the GPU resource quantity, and changes dynamically within the selected time interval. The pre-training start and end time nodes in this embodiment are used as a basis for regulating the start and stop states of the pre-training task of the large language model. There may be multiple parameters for characterizing the pre-training start and end time nodes in this embodiment, which will be exemplified later. The GPU resource quantity control parameters that can be used in this embodiment will also be exemplified later and are not limited here.

[0037] After completing the parameter setting process in step 101, the pre-training task for the large language model can be started. Referring to Figure 1, in step 102, the relevant parameter values ​​can be determined in step 101, and the task start and stop status of the large language model pre-training task can be adjusted within the selected time interval, and the amount of GPU resources used for pre-training at each time node during the pre-training period can be adjusted. It should be understood that in actual applications, the amount of GPU resources used for pre-training will not be provided during the non-pre-training period.

[0038] In this way, within the selected time interval, the GPU resource control parameter values ​​at the pre-training start and end time nodes and each time node during the pre-training period can be combined to dynamically control the amount of GPU resources used for pre-training of the large language model at different time nodes. This control can ensure that the total estimated carbon emissions corresponding to pre-training of the large language model within the selected time interval meet the specified requirements.

[0039] In summary, in this embodiment, for the pre-training process of the large language model, based on the dynamically changing carbon emission intensity within the selected time interval, the pre-training start and end time nodes within the time interval and the GPU resource quantity control parameter values ​​at each time node during the pre-training period are determined for the large language model, so that the total estimated pre-training carbon emissions corresponding to the large language model within the time interval meet the specified requirements; based on this, the GPU resource quantity used for pre-training of the large language model can be dynamically controlled within the time interval according to the pre-training start and end time nodes and each GPU resource quantity control parameter value determined above. In this way, within the selected time interval, the pre-training start and end time nodes and the GPU resource quantity control parameter values ​​at each time node during the pre-training period can be reasonably controlled based on the dynamically changing carbon emission intensity, thereby dynamically controlling the GPU resource quantity used for pre-training of the large language model, and then reasonably dividing the pre-training tasks into relevant time nodes within the selected time interval, which can effectively reduce the carbon emissions generated by the pre-training of the large language model.

[0040] In the above or following embodiments, in step 101, the problem of determining the pre-training start and end time nodes within the time interval and the GPU resource quantity control parameter values ​​at each time node during the pre-training for a large language model can be converted into a problem of solving the objective function by constructing an objective function.

[0041] To this end, in step 101, the objective function can be constructed with the goal of achieving the specified requirements for the total estimated pre-training carbon emissions of the large language model within the selected time interval, wherein the objective function uses the product of the carbon emission intensity and the total energy consumption at a single time node in the time interval to represent the estimated carbon emissions at the corresponding time node, and the total energy consumption at a single time node is affected by the GPU resource quantity control parameters and the parameters used to characterize the pre-training start and end time nodes; the objective function is solved to determine the pre-training start and end time nodes and the values ​​of each GPU resource quantity control parameter.

[0042] As mentioned above, in this embodiment, the carbon emission intensity at a single time point can refer to the carbon emissions required to produce one kilowatt-hour of electricity, while the total energy consumption at a single time point in the objective function can refer to the number of kilowatt-hours of electricity consumed by the large language model at that time point. Thus, the product of the two can represent the estimated carbon emissions at a single time point. Thus, in the objective function, the estimated carbon emissions at each time point during the pre-training period can be summed to represent the estimated total pre-trained carbon emissions of the large language model for the selected time interval.

[0043] In addition, the parameters to be solved in the objective function may include parameters used to characterize the start and end time nodes of pre-training and parameters for controlling GPU resource consumption. These two parameters act as factors affecting the total energy consumption at a single time node in the objective function. Solving the objective function thus yields the corresponding values ​​for these parameters.

[0044] Two optional GPU resource control parameters are provided below, and the resource optimization and carbon emission reduction solutions for large language models provided by this embodiment are described with respect to the two GPU resource control parameters.

[0045] 1. The GPU resource control parameters in this embodiment may include GPU parallelism.

[0046] The GPU parallelism may refer to the number of GPUs running at a single time node.

[0047] In this case, in an exemplary scheme for constructing an objective function: a first energy consumption value caused by a single GPU in a running state at a single time node and a second energy consumption value caused by a single GPU in an idle state at a single time node can be obtained; based on the GPU parallelism and the first energy consumption value and the second energy consumption value at a single time node, the energy consumption generated on the GPU at the corresponding time node in the time interval is characterized; combined with the parameters used to characterize the start and end time nodes of pre-training and the characterized energy consumption generated on the GPU at a single time node, the total energy consumption at the corresponding time node is characterized to construct an objective function.

[0048] This exemplary solution characterizes the total energy consumption at a single time node based on GPU parallelism and parameters used to characterize the pre-training start and end time nodes. This allows the objective function to reflect the impact of GPU parallelism and the pre-training start and end time nodes on total energy consumption. This solution introduces a first energy consumption value at a single time node when a single GPU is in a running state, and a second energy consumption value at a single time node when a single GPU is in an idle state.

[0049] Based on this, in this exemplary solution, an exemplary expression of the objective function may be:

[0050] Among them, C t represents the carbon emission intensity at time node t, n t Indicates the GPU parallelism at time node t, (P G +P active ) represents the first energy consumption value, P G Indicates the energy consumption value of the GPU at a single time node when it is in operation. activeIt represents the energy consumption value of other parts of the server at a single time node when the GPU is in operation; (Nn t ) represents the number of idle GPUs at time node t, N is the total number of GPUs, and accordingly, (P idle +P Gi ) represents the second energy consumption value, P Gi Indicates the energy consumption value of the GPU at a single time node when it is idle. idle It represents the energy consumption value of a single time node caused by other parts of the server where the GPU is located when the GPU is in idle state; u t and y t is a parameter used to characterize the start and end time nodes of pre-training, where u t Indicates whether the time node t is in the pre-training period, y t Indicates whether the server startup event occurs at time node t due to the start of pre-training, K*y t represents the power-on energy consumption value caused by the server power-on event; T represents the task delay corresponding to the start and end time nodes of the pre-training.

[0051] The above P G and P active You can use empirical values. For different models of GPU, P G The value may be different, P active The values ​​may also be different. In addition, P G and P active The ratio between them is usually γ:1. In practical applications, P G The value is an empirical value, and by setting γ, P is calculated. active In this case, the value of γ can be set as needed based on factors such as the GPU model. For example, γ can be set to 3, etc., and this is not limited here.

[0052] In addition, in practical applications, u t The value of u can be 0 or 1. t 1 means that the time node t is in the pre-training period, u t If it is 0, it means that the time node t is not in the pre-training period; t The value of y is 0 or 1. t 1 indicates that a server startup event occurred at time node t due to the start of pre-training, y t If it is 0, it means that no server startup event occurs at time node t.

[0053] It should be understood that the above expression of the objective function is exemplary, and other expressions can be used in this embodiment to reflect the impact of GPU parallelism and pre-training start and end time nodes on the total energy consumption. For example, the representation of the first energy consumption value can consider P G There is no need to consider P active For another example, the characterization of the second energy consumption value may take into account P Gi There is no need to consider P idle For example, in addition to the aforementioned energy consumption value of the server itself, the energy consumption value of other parts of the server, and the power-on energy consumption value, more energy consumption values ​​of other dimensions that may be affected by GPU parallelism or pre-training start and end time nodes can be added to integrate the energy consumption values ​​of multiple dimensions to represent the total energy consumption at a single time node. For example, there is no need to use u t The task start and stop states are characterized at each time point. Instead of using T as the upper limit for t, multiple values ​​of t can be directly limited. In other words, each time point during pre-training is used as the value of t in the objective function. Further examples are not given here.

[0054] As can be seen, in this embodiment, the objective function can take into account the energy consumption values ​​of various dimensions that may be caused by executing large language model pre-training at a single time node, and reflect the impact of GPU parallelism or the pre-training start and end time nodes on the energy consumption values ​​of the corresponding dimensions, so as to characterize the total energy consumption at a single time node. In this way, by solving the objective function in combination with the dynamically changing carbon emission intensity within the selected time interval, the GPU parallelism at the pre-training start and end time nodes within the time interval and the pre-training time nodes can be determined for the large language model.

[0055] On this basis, the GPU parallelism can be dynamically adjusted within the selected time interval according to the solved pre-training start and end time nodes and the GPU parallelism at each time node during pre-training to achieve GPU resource control.

[0056] 2. The GPU resource control parameters in this embodiment may include the frequency and voltage of the GPU.

[0057] The frequency of the GPU may refer to the GPU main frequency, i.e., the clock frequency. The clock frequency refers to the frequency at which the processor's clock generator generates pulses, which are used to synchronize the operation of processor components and are used as an indicator of processor speed. It is measured in Hertz (Hz). Generally speaking, a higher frequency means a faster computing speed of the processor. The GPU voltage refers to the voltage allocated to a single GPU at a single time node. For ease of calculation, in this embodiment, at the same time node, it can be assumed that all GPUs in the computing system are running at the same frequency and voltage.

[0058] In this case, an exemplary scheme for constructing the objective function is as follows: based on the GPU frequency and voltage at a single time node, the energy consumption generated on the GPU at a single time node is characterized; combined with the parameters used to characterize the start and end time nodes of pre-training and the characterized energy consumption generated on the GPU at a single time node, the total energy consumption at the corresponding time node is characterized to construct the objective function.

[0059] Optionally, in this exemplary embodiment, the relationship between the energy consumption of the GPU at a single time node and the GPU frequency and voltage can be determined according to the principle of dynamic voltage and frequency scaling (DVFS). Based on this relationship, the energy consumption of the GPU at a single time node is characterized in the objective function based on the GPU frequency and voltage at the single time node. Under the principle of dynamic voltage and frequency scaling, the energy consumption of the GPU at a single time node is affected by both the GPU frequency and voltage and is directly proportional to both the GPU frequency and voltage.

[0060] In this exemplary scheme, it is proposed to characterize the total energy consumption at a single time node based on the GPU frequency and voltage and the parameters used to characterize the start and end time nodes of pre-training, so as to reflect the impact of the GPU frequency and voltage and the start and end time nodes of pre-training on the total energy consumption in the objective function.

[0061] Based on this, in this exemplary solution, an exemplary expression of the objective function may be:

[0062] and

[0063] Among them, f t Indicates the GPU frequency at time node t, V t represents the GPU voltage at time node t, N represents the total number of GPUs; P t Represents the energy consumption value of a single GPU at time node t, P active,t It represents the energy consumption value caused by other parts of the server where a single GPU is located at time node t, and γ represents P t and P active,t The proportional relationship between u t and y t is a parameter used to characterize the start and end time nodes of pre-training, where u t Indicates whether the time node t is in the pre-training period, y t Indicates whether the server startup event occurs at time node t due to the start of pre-training, K*y t represents the power-on energy consumption value caused by the server power-on event; T represents the task delay corresponding to the start and end time nodes of pre-training, and aC can be regarded as a constant.

[0064] Similarly, in practical applications, u t The value of u can be 0 or 1. t 1 means that the time node t is in the pre-training period, u t If it is 0, it means that the time node t is not in the pre-training period; t The value of y is 0 or 1. t 1 indicates that a server startup event occurred at time node t due to the start of pre-training, y t If it is 0, it means that no server startup event occurs at time node t.

[0065] In this exemplary expression, according to the principle of dynamic voltage and frequency adjustment, the relationship between the energy consumption generated on the GPU at time node t and the GPU frequency and voltage is represented as Based on P t and P active,t The proportional relationship between them is that P can be active,t Characterized by:

[0066] This changes the objective function to use the GPU frequency and voltage as well as the parameters used to characterize the start and end time nodes of pre-training as the parameters to be solved.

[0067] It should be understood that the above expression of the objective function is exemplary, and other expressions can be used in this embodiment to reflect the impact of GPU frequency and voltage and pre-training start and end time nodes on the total energy consumption. For example, consider P t There is no need to consider P active,t For another example, in addition to the aforementioned energy consumption value, the energy consumption value generated by other parts of the server, and the power-on energy consumption value, more energy consumption values ​​of other dimensions that may be affected by GPU frequency, GPU voltage, or pre-training start and end time nodes can be added to integrate energy consumption values ​​of multiple dimensions to represent the total energy consumption at a single time node. For another example, there is no need to use u t The task start and stop states are characterized at each time point. Instead of using T as the upper limit for t, multiple values ​​of t can be directly limited. In other words, each time point during pre-training is used as the value of t in the objective function. Further examples are not given here.

[0068] As can be seen, in this embodiment, the objective function can take into account the energy consumption values ​​of various dimensions that may be caused by executing large language model pre-training at a single time node, and reflect the impact of GPU frequency, GPU voltage, or pre-training start and end time nodes on the energy consumption values ​​of corresponding dimensions, so as to characterize the total energy consumption at a single time node. In this way, by solving the objective function in combination with the dynamically changing carbon emission intensity within the selected time interval, the GPU frequency and voltage at the pre-training start and end time nodes within the time interval, as well as at each time node during pre-training, can be determined for the large language model.

[0069] On this basis, the GPU frequency and voltage can be dynamically adjusted within the selected time interval according to the solved pre-training start and end time nodes and the GPU frequency and voltage at each time node during pre-training to achieve GPU resource control.

[0070] In this embodiment, the above two exemplary GPU resource control parameters are provided. It should be understood that this embodiment may also adopt other types of GPU resource control parameters to dynamically adjust the GPU resource amount provided to the large language model within a selected time interval.

[0071] Furthermore, in this embodiment, in order to improve the accuracy of the pre-training start and end time nodes solved for the large language model in this exemplary solution and the GPU parallelism at each time node during pre-training, constraints can also be set for the objective function.

[0072] In this embodiment, the constraints set for the objective function may include but are not limited to the task delay corresponding to the pre-training start and end time nodes not exceeding a preset delay threshold; and the amount of tasks that can be completed within the time interval is not less than the specified total amount of tasks, etc.

[0073] The task latency corresponding to the pre-training start and end time nodes corresponds to T in the aforementioned exemplary expression of the objective function, while the preset latency threshold refers to the maximum latency allowed by the large language model. By ensuring that the task latency corresponding to the pre-training start and end time nodes does not exceed the preset latency threshold, this constraint allows for more reasonable determination of the pre-training start and end time nodes, ensuring that the large language model is completed on time.

[0074] In addition, the inventors discovered during the research process that there is a linear functional relationship between the energy consumption value of a single GPU at a single time node and the amount of tasks it can complete.

[0075] When the GPU resource control parameters include GPU parallelism, this linear function relationship can be expressed as: t =k1*P G +b, and in this case, the amount of tasks that can be completed after GPU resource regulation can be represented as:

[0076] In this way, the amount of tasks that can be completed after GPU resource regulation for the large language model according to the GPU parallelism at each time node is not less than the specified total amount of tasks. This constraint can be expressed as:

[0077] Among them, S 总 Represents the total number of tasks specified above. It can be seen that in this case, this constraint is related to the GPU parallelism. Therefore, based on this constraint, the value of the GPU parallelism can be constrained, making the final value of the GPU parallelism more reasonable.

[0078] When the GPU resource control parameters include GPU parallelism, this linear function relationship can be expressed as: t =k2*P t +b and P t =k3*f t +b, and in this case, the amount of tasks that can be completed after GPU resource regulation can be represented as:

[0079] In this way, the amount of tasks that can be completed after GPU resource regulation for the large language model based on the GPU frequency and voltage at each time node is not less than the specified total amount of tasks. This constraint can be expressed as:

[0080] It can be seen that in this case, the constraint condition is related to the frequency of the GPU. Therefore, based on the constraint condition, the value of the GPU frequency can be constrained to make the final value of the GPU frequency more reasonable.

[0081] Of course, in addition to the two exemplary constraints mentioned above, in this embodiment, other constraints may also be set for the objective function.

[0082] For example, when the GPU resource control parameter includes GPU parallelism, the GPU parallelism at each time node during pre-training can be set to not be higher than the total number of GPUs. t ≤n t ≤N*u t ,u t When n is 0, t Constrained to 0.

[0083] For another example, when the GPU resource control parameter includes GPU parallelism, the energy consumption value of a single GPU at a single time node during pre-training can be set to be within the upper and lower limits of the energy consumption supported by the GPU. min *u t ≤Pt ≤P max *u t , where u t When P is 0, t Constrained to 0.

[0084] For another example, when the GPU resource control parameters include GPU parallelism, the GPU frequency at a single time node can be set to be within the upper and lower limits of the frequency supported by the GPU, and the voltage at a single time node can be set to be within the upper and lower limits of the voltage supported by the GPU. min ≤f t ≤f max ; V min ≤V t ≤V max .

[0085] In addition, regardless of the parameters used for GPU resource control, the constraints set for the objective function can also include that the pre-training start and end time nodes are consistent with the start and stop timing of the GPU server, and that the task start and stop occur once within the time interval. This constraint can be specifically characterized as:

[0086] And, u t -u t-1 =y t -y t-1 ,t≥1

[0087] Among them, z t It can indicate whether the server shutdown event occurred at time node t due to the end of pre-training, z t The value of can be 0 or 1, where z t 1 indicates that a server shutdown event occurred at time node t, z t A value of 0 indicates that no server shutdown event occurred at time t. This constraint ensures that pre-training starts and ends once, and that the server starts and stops once, within the selected time interval, and that the two remain consistent. This constraint effectively constrains the pre-training start and end times.

[0088] It is worth noting that the constraints in this embodiment are not limited to this, and other constraints can be set without being limited to this. No further examples are given here.

[0089] Thus, in this embodiment, the objective function can be solved based on the constraints to obtain a solution result, which will include the parameter values ​​corresponding to the parameters used to characterize the pre-training start and end time nodes, as well as the GPU resource quantity control parameter values ​​at each time node during the pre-training period. Through these constraints, the pre-training start and end time nodes within the time interval and the GPU resource quantity control parameter values ​​at each time node during the pre-training period can be more reasonably determined for the large language model, thereby more reasonably controlling the GPU resource quantity and achieving better carbon emission reduction effects.

[0090] It should be noted that some of the processes described in the above embodiments and the accompanying drawings include multiple operations that appear in a specific order, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are used to distinguish between different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different application ends, energy consumption values, etc., and do not represent the order of precedence, nor do they limit "first" and "second" to be different types.

[0091] FIG3 is a schematic diagram of a computing device according to another exemplary embodiment of the present invention. As shown in FIG3 , the computing device includes a memory 30 and a processor 31 .

[0092] The processor 31 is coupled to the memory 30 and is configured to execute the computer program in the memory 30 to:

[0093] Select a time interval for pre-training the large language model to be processed;

[0094] Based on the dynamically changing carbon emission intensity within the time interval, determining the pre-training start and end time nodes within the time interval and the GPU resource control parameter values ​​at each time node during the pre-training for the large language model, so that the estimated total pre-training carbon emissions corresponding to the large language model within the time interval meet the specified requirements;

[0095] According to the pre-training start and end time nodes and the respective GPU resource control parameter values, the GPU resource amount used for pre-training the large language model is dynamically controlled within the time interval.

[0096] In an optional embodiment, when the processor 31 determines the pre-training start and end time nodes within the time interval and the GPU resource amount control parameter values ​​at each time node during the pre-training for the large language model based on the dynamically changing carbon emission intensity within the time interval, the setting is:

[0097] An objective function is constructed with the goal of ensuring that the total estimated pre-training carbon emissions of the large language model within the time interval meet specified requirements. The objective function represents the estimated carbon emissions at a corresponding time node by multiplying the carbon emission intensity and the total energy consumption at a single time node within the time interval. Furthermore, the total energy consumption at a single time node is influenced by a GPU resource control parameter and a parameter used to represent the start and end time nodes of pre-training.

[0098] Solve the objective function to determine the pre-training start and end time nodes and the respective GPU resource control parameter values.

[0099] In an optional embodiment, the GPU resource control parameter includes GPU parallelism. When the processor 31 constructs an objective function with the goal of achieving a specified requirement for the total estimated pre-trained carbon emissions corresponding to the large language model within the time interval, the objective function is set to:

[0100] Obtaining a first energy consumption value caused at a single time node when a single GPU is in a running state and a second energy consumption value caused at a single time node when the single GPU is in an idle state;

[0101] Characterizing energy consumption generated on the GPU at a corresponding time node in the time interval based on the GPU parallelism at a single time node and the first energy consumption value and the second energy consumption value;

[0102] The objective function is constructed by combining the parameters used to characterize the start and end time nodes of the pre-training and the energy consumption generated on the GPU at a single time node. The total energy consumption at the corresponding time node is characterized.

[0103] In an optional embodiment, the objective function can be represented as:

[0104] Among them, C t represents the carbon emission intensity at time node t, n t Indicates the GPU parallelism at time node t, (P G +P active ) represents the first energy consumption value, P G Indicates the energy consumption value of the GPU at a single time node when it is in operation. active It represents the energy consumption value of other parts of the server at a single time node when the GPU is in operation; (Nn t ) represents the number of idle GPUs at time node t, N is the total number of GPUs, and accordingly, (P idle +P Gi ) represents the second energy consumption value, PGi Indicates the energy consumption value of the GPU at a single time node when it is idle. idle It represents the energy consumption value of other parts of the server at a single time node when the GPU is in idle state; u t and y t is a parameter used to characterize the start and end time nodes of pre-training, where u t Indicates whether the time node t is in the pre-training period, y t Indicates whether the server startup event occurs at time node t due to the start of pre-training, K*y t represents the power-on energy consumption value caused by the server power-on event; T represents the task delay corresponding to the start and end time nodes of the pre-training.

[0105] In an optional embodiment, the GPU resource control parameters include the frequency and voltage of the GPU. When the processor 31 constructs an objective function with the goal of achieving a specified requirement for the total estimated pre-trained carbon emissions corresponding to the large language model within the time interval, the objective function is set to:

[0106] Based on the GPU frequency and voltage at a single time node, the energy consumption generated by the GPU at a single time node is characterized;

[0107] The objective function is constructed by combining the parameters used to characterize the start and end time nodes of the pre-training and the energy consumption generated on the GPU at a single time node. The total energy consumption at the corresponding time node is characterized.

[0108] In an optional embodiment, the objective function is characterized as:

[0109] and

[0110] Among them, C t represents the carbon emission intensity at time node t, f t Indicates the GPU frequency at time node t, V t represents the GPU voltage at time node t, N represents the total number of GPUs; P t Represents the energy consumption value of a single GPU at time node t, P active,t It represents the energy consumption of other parts of a single GPU on the server at time node t, and γ represents P t and P active,t The proportional relationship between u t and y t is a parameter used to characterize the start and end time nodes of pre-training, where u t Indicates whether the time node t is in the pre-training period, y tIndicates whether the server startup event occurs at time node t due to the start of pre-training, K*y t represents the power-on energy consumption value caused by the server power-on event; T represents the task delay corresponding to the start and end time nodes of the pre-training.

[0111] In an optional embodiment, when solving the objective function, the processor 31 is set to:

[0112] Configuring constraints for the objective function, wherein the constraints include that the task delay corresponding to the pre-training start and end time nodes does not exceed a preset delay threshold; and the amount of tasks that can be completed within the selected time interval is not less than the specified total amount of tasks;

[0113] The objective function is solved based on the constraint conditions to obtain a solution result, which includes parameter values ​​corresponding to parameters used to characterize the start and end time nodes of pre-training and GPU resource quantity control parameter values ​​at each time node during pre-training.

[0114] In an optional embodiment, the constraint condition also includes that the pre-training start and end time nodes are consistent with the start and stop timing of the server where the GPU is located, and a task start and stop occurs once within the time interval.

[0115] In an alternative embodiment, u t The value of u is 0 or 1. t 1 means that the time node t is in the pre-training period, u t If it is 0, it means that the time node t is not in the pre-training period; t The value of y is 0 or 1. t 1 indicates that a server startup event occurred at time node t due to the start of pre-training, y t If it is 0, it means that no server startup event occurs at time node t.

[0116] Furthermore, as shown in Figure 3, the computing device also includes other components such as a communication component 32 and a power supply component 33. Figure 3 schematically shows some components, which does not mean that the computing device cannot include other components besides those shown in Figure 3.

[0117] It is worth noting that the technical details in the above-mentioned embodiments of the computing device can be referred to the relevant description in the aforementioned method embodiment. In order to save space, they will not be repeated here, but this should not cause any loss of the scope of protection of this application.

[0118] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement the steps that can be executed by a computing device in the above method embodiment.

[0119] The memory in FIG. 3 is configured to store computer programs and may be configured to store various other data to support operations on the computing platform. Examples of such data include instructions for any application or method operating on the computing platform, contact data, phone book data, messages, images, videos, and the like. The memory may be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0120] The communication component in Figure 3 above is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0121] The power supply assembly in Figure 3 provides power to various components of the device in which the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply assembly is located.

[0122] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0123] The present application is described with reference to the flow chart and / or block diagram of the method, device (system), and computer program product according to the embodiment of the present application. It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow process and / or box in the flow chart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processing machine or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one flow chart flow or multiple flows and / or one box or multiple boxes of the block diagram.

[0124] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0125] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0126] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0127] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0128] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media configured to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0129] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0130] The foregoing is an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included in the protection scope of the present application.

Claims

1. A resource optimization and carbon emission reduction method for a large language model, comprising: Select a time interval for pre-training the large language model to be processed; Based on the dynamically changing carbon emission intensity within the time interval, determine the pre-training start and end time nodes within the time interval and the GPU resource amount control parameter value during the pre-training for the large language model, so that the estimated total amount of pre-training carbon emissions corresponding to the large language model within the time interval meets the specified requirements; According to the pre-training start and end time nodes and the values ​​of the respective GPU resource quantity control parameters, the amount of GPU resources used for pre-training the large language model is dynamically controlled within the time interval.

2. The method according to claim 1, based on the dynamically changing carbon emission intensity in the time interval, determining the pre-training start and end time nodes in the time interval and the GPU resource quantity control parameter values ​​at each time node during the pre-training for the large language model, comprising: An objective function is constructed with the goal that the total estimated pre-trained carbon emissions corresponding to the large language model in the time interval reaches the specified requirements, wherein the objective function uses the product of the carbon emission intensity and the total energy consumption at a single time node in the time interval to represent the estimated carbon emissions at the corresponding time node, and the total energy consumption at a single time node uses the GPU resource control parameter and the parameter used to represent the pre-training start and end time nodes as influencing factors; Solve the objective function to determine the start and end time nodes of the pre-training and the value of each GPU resource quantity control parameter.

3. According to the method of claim 2, the GPU resource quantity control parameter includes GPU parallelism, and the objective function is constructed with the goal that the total estimated amount of pre-trained carbon emissions corresponding to the large language model in the time interval reaches the specified requirement, including: Obtain a first energy consumption value caused at a single time node when a single GPU is in a running state and a second energy consumption value caused at a single time node when a single GPU is in an idle state; Characterizing the energy consumption caused by the GPU at the corresponding time node in the time interval based on the GPU parallelism at a single time node, the first energy consumption value, and the second energy consumption value; Combined with the parameters used to characterize the start and end time nodes of the pre-training and the energy consumption caused by the GPU at a single time node, the total energy consumption at the corresponding time node is characterized to construct the objective function.

4. According to the method of claim 3, the objective function is characterized as: in, C t represents the carbon emission intensity at time node t, n t represents the GPU parallelism at time node t, (P G +P active ) represents the first energy consumption value, P G Indicates the energy consumption value of the GPU at a single time node when it is in operation, P active It represents the energy consumption value of a single time node caused by other parts of the server where a GPU is located when it is in operation; (Nn t ) represents the number of idle GPUs at time node t, N is the total number of GPUs, and accordingly, (P idle +P Gi ) represents the second energy consumption value, P Gi Indicates that the GPU is idle at a single time node The self-energy consumption value under idle It represents the energy consumption value of a single time node caused by other parts of the server where a GPU is located when it is idle; u t and t is a parameter used to characterize the start and end time nodes of pre-training, where u t Indicates whether the time node t is in the pre-training period, y t Indicates whether the server startup event occurs at time node t due to the start of pre-training, K*y t represents the power-on energy consumption value caused by the server power-on event; T represents the task delay corresponding to the start and end time nodes of the pre-training.

5. According to the method of claim 2, the GPU resource control parameters include the frequency and voltage of the GPU, and the objective function is constructed with the goal that the total estimated pre-trained carbon emissions corresponding to the large language model in the time interval reaches the specified requirement, including: Based on the GPU frequency and voltage at a single time node, the energy consumption generated on the GPU at a single time node is characterized; The total energy consumption at the corresponding time node is characterized by combining the parameters used to characterize the start and end time nodes of the pre-training and the energy consumption generated on the GPU at the characterized single time node to construct the objective function.

6. According to the method of claim 5, the objective function is characterized as: and in, C t represents the carbon emission intensity at time node t, f t Represents the GPU frequency at time node t, V t represents the GPU voltage at time node t, N represents the total number of GPUs; P t Represents the energy consumption value of a single GPU at time node t, P active,t It represents the energy consumption of a single GPU at time node t caused by other parts of the server where it is located, and γ represents P t and P active,t The proportional relationship between t and t is a parameter used to characterize the start and end time nodes of pre-training, where u t Indicates whether the time node t is in the pre-training period, y t Indicates whether the server startup event occurs at time node t due to the start of pre-training, K*y t represents the power-on energy consumption value caused by the server power-on event; T represents the task delay corresponding to the start and end time nodes of the pre-training.

7. The method according to claim 2, solving the objective function comprises: Configuring constraint conditions for the objective function, wherein the constraint conditions include that the task delay corresponding to the pre-training start and end time nodes does not exceed a preset delay threshold; And the amount of tasks that can be completed within the time interval is not less than the total amount of specified tasks; The objective function is solved based on the constraint conditions to obtain a solution result, wherein the solution result includes parameter values ​​corresponding to parameters used to characterize the start and end time nodes of the pre-training and the GPU resource quantity control parameter values ​​at each time node during the pre-training period.

8. According to the method of claim 7, the constraint conditions also include that the start and end time nodes of the pre-training are consistent with the start and stop timing of the server where the GPU is located, and a task start and stop occurs once within the time interval.

9. The method according to any one of claims 4 or 6, t The value of u is 0 or 1. t 1 means that time node t is in the pre-training period, u t If it is 0, it means that the pre-training period is not in progress at time node t; t The value of y is 0 or 1. t 1 indicates that a server startup event occurred at time node t due to the start of pre-training. t If it is 0, it means that no server startup event occurred at time node t.

10. A computing device comprising a memory and a processor; The memory is configured to store one or more computer instructions; The processor is coupled to the memory and is configured to execute the one or more computer instructions for executing the resource optimization and carbon emission reduction method for a large language model as described in any one of claims 1-9.

11. A computer-readable storage medium storing computer instructions, which, when executed by one or more processors, causes the one or more processors to execute the resource optimization and carbon emission reduction method for a large language model as described in any one of claims 1-9.

Images