Large model energy consumption optimization method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a large model energy consumption optimization method and device, computer equipment, a computer readable storage medium and a computer program product. Comprising the steps of collecting hardware state data and task data of a large model; performing stage identification according to the task data, and determining a current stage; performing state prediction according to the hardware state data and the task data through a prediction model corresponding to the current stage to obtain target state information; generating optimization parameters of the current stage through a joint optimizer according to the target state information; and adjusting the resources of the large model according to the optimization parameters. In combination with stage perception and dynamic resource adjustment, a differentiated resource adjustment strategy based on different stages is realized, specific energy consumption pain points in an AI scene are solved, high power consumption of large model training, instantaneous fluctuation of reasoning requests and the like are reduced, sustainable and efficient operation of an AI system is ensured, the computing power demand and resource consumption of a large model are balanced, and the system performance is improved. And wide landing and development of a large model in various scenes are facilitated.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, computer device, computer-readable storage medium, and computer program product for large-scale model energy consumption optimization. Background Technology

[0002] The training, fine-tuning and inference phases of large model operations exhibit significant differences in resource requirements. Traditional solutions mostly adopt a "one-size-fits-all" strategy, such as fixed power capping. During the training phase, excessive current limiting may increase convergence latency by 20%, while during the inference phase, response lag may lead to a decline in service quality.

[0003] Therefore, there is an urgent need for an energy consumption optimization algorithm that can adaptively adjust according to the operation stage in order to break through the bottleneck of traditional solutions and achieve the sustainable and efficient operation of AI (Artificial Intelligence) systems. Summary of the Invention

[0004] Therefore, it is necessary to provide a large-scale energy consumption optimization method, device, computer equipment, computer-readable storage medium, and computer program product to address the above-mentioned technical problems. This method can combine stage perception and dynamic resource adjustment to realize differentiated resource adjustment strategies based on different stages.

[0005] Firstly, this application provides a method for optimizing energy consumption in large-scale models, including:

[0006] Collect hardware status data and task data from large models;

[0007] Stage identification is performed based on task data to determine the current stage;

[0008] The target state information is obtained by using the prediction model corresponding to the current stage to predict the state based on hardware state data and task data.

[0009] The joint optimizer generates the optimization parameters for the current stage based on the target state information.

[0010] Based on the optimization parameters, the resources of the large model are adjusted.

[0011] In one embodiment, the current stage includes a training stage, where the prediction model is an energy consumption prediction model, and the target state information predicted by the energy consumption prediction model is the target power. The optimization parameters include the optimal batch size and the frozen network layer parameters. The joint optimizer generates the optimization parameters for the current stage based on the target state information, including:

[0012] The adjustment factor is determined by the joint optimizer based on the current power and the target power;

[0013] Determine the optimal batch size based on the adjustment factor and the current batch size;

[0014] Identify network layers in a large model whose contribution value is less than a set value;

[0015] Based on the identified network layers, determine the parameters for freezing the network layers;

[0016] Accordingly, the resources of the large model are adjusted based on the optimization parameters, including:

[0017] Adjust the training parameters of the large model based on the optimal batch size;

[0018] Based on the parameters of the frozen network layers, the corresponding network layers in the large model are frozen.

[0019] In one embodiment, the current stage includes a fine-tuning stage. The prediction model for the fine-tuning stage is an energy consumption prediction model. The target state information predicted by the energy consumption prediction model is the peak power. The optimization parameters include the gradient accumulation step count and the fine-tuning strategy setting parameters. The joint optimizer generates the optimization parameters for the current stage based on the target state information, including:

[0020] The joint optimizer determines the number of gradient accumulation steps based on the peak power and the current power.

[0021] Use the PEFT parameter selector to determine the fine-tuning strategy settings parameters;

[0022] Accordingly, the resources of the large model are adjusted based on the optimization parameters, including:

[0023] Add gradient accumulation steps to the fine-tuning phase of the large model, and adjust the fine-tuning strategy of the large model according to the parameters set according to the fine-tuning strategy.

[0024] In one embodiment, the current stage includes an inference stage, where the prediction model is a load prediction model. The target state information predicted by the load prediction model is the request rate, and the optimization parameters include target accuracy and request queue configuration parameters. The joint optimizer generates the optimization parameters for the current stage based on the target state information, including:

[0025] The joint optimizer selects the target accuracy from multiple preset large model accuracies based on the request rate and determines the request queue configuration parameters.

[0026] Accordingly, the resources of the large model are adjusted based on the optimization parameters, including:

[0027] Adjust the accuracy of the large model to the target accuracy;

[0028] Configure the request processing mechanism of the request queue according to the request queue configuration parameters; the request processing mechanism includes setting priorities for requests in the request queue and allocating computing resources to high-priority requests first.

[0029] In one embodiment, after adjusting the resources of the large model according to optimization parameters, the method further includes:

[0030] Based on the Hungarian algorithm, the tasks of the current stage are assigned to the corresponding optimal hardware for processing;

[0031] The hardware voltage or frequency is dynamically adjusted based on the task execution status.

[0032] In one embodiment, the method further includes:

[0033] Collect measured status information;

[0034] The prediction error is determined based on the measured state information and the target state information;

[0035] If the prediction error exceeds a preset value, the prediction model corresponding to the current stage will be trained and updated.

[0036] Secondly, this application also provides a large-scale model energy consumption optimization device, comprising:

[0037] The input module is used to collect hardware status data and task data for large models;

[0038] The stage identification module is used to identify the current stage based on task data.

[0039] The state prediction module is used to predict the state based on hardware state data and task data using the prediction model corresponding to the current stage, and obtain the target state information.

[0040] The optimization analysis module is used to generate optimization parameters for the current stage based on the target state information through a joint optimizer;

[0041] The phase optimization module is used to adjust the resources of the large model based on the optimization parameters.

[0042] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in the first aspect above.

[0043] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0044] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect above.

[0045] The aforementioned large-scale model energy consumption optimization method, device, computer equipment, computer-readable storage medium, and computer program product collect hardware status data and large-scale model task data; identify the current stage based on the task data; predict the target state information using the prediction model corresponding to the current stage based on the hardware status data and task data; generate optimization parameters for the current stage using a joint optimizer based on the target state information; and adjust the resources of the large-scale model according to the optimization parameters. By combining stage awareness with dynamic resource adjustment, this approach enables differentiated resource adjustment strategies based on different stages, addressing the unique energy consumption challenges in AI scenarios. It reduces the high power consumption of large-scale model training and the instantaneous fluctuations in inference requests, ensuring the sustainable and efficient operation of AI systems, balancing the computing power requirements and resource consumption of large-scale models, and facilitating the widespread deployment and development of large-scale models in various scenarios. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating a large-scale model energy consumption optimization method in one embodiment;

[0048] Figure 2 This is a schematic diagram of the system architecture in one embodiment;

[0049] Figure 3 This is a flowchart illustrating the large-scale model energy consumption optimization method in another embodiment;

[0050] Figure 4 This is a structural block diagram of a large-scale model energy consumption optimization device in one embodiment;

[0051] Figure 5 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0054] It is understood that the method provided in this application embodiment can be applied to computer devices with reasoning capabilities, such as terminals or servers. The method can also be applied to systems including terminals and servers, and implemented through interaction between the terminals and servers. Terminals can be, but are not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. Head-mounted devices can be virtual reality (VR) devices, augmented reality (AR) devices, smart glasses, etc. Servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing cloud computing services.

[0055] In one exemplary embodiment, such as Figure 1 As shown, a method for optimizing the energy consumption of a large model is provided, which includes:

[0056] Step 102: Collect hardware status data and task data for the large model.

[0057] Hardware status data refers to various measurements and information that reflect the current operating status, performance, and physical conditions of each piece of computer hardware in the hardware layer. This information can be collected by hardware sensors (sampling frequency ≥ 1kHz) deployed in the hardware layer, including data such as processor temperature, voltage, and memory bandwidth. The processor can be a GPU (Central Processing Unit), a TPU (Tensor Processing Unit), or a CPU (Graphics Processing Unit), etc.

[0058] The task data for large models refers to training data, fine-tuning instructions, or inference requests. In specific implementations, relevant task features are extracted for the task data of large models, such as batch size, sequence length, and gradient update frequency.

[0059] In one alternative implementation, historical energy consumption logs are also collected, which include energy consumption data from past tasks, for use in training the prediction model.

[0060] Step 104: Identify the current stage based on the task data.

[0061] The current stage refers to the training stage, fine-tuning stage, or inference stage. Large-scale model training refers to the process of optimizing model parameters using a large amount of data to learn data features. The goal of training is to find a set of parameters that minimizes the difference between the predicted and actual values ​​of the large model. Large-scale model fine-tuning refers to the process of further training the model on a domain-specific dataset, building upon a pre-trained model. Pre-trained models are typically trained on large-scale, general-purpose data, learning broad knowledge and language patterns, but this knowledge often lacks specificity. For example, in natural language processing, a pre-trained model may not accurately understand domain-specific terminology or logic. Through fine-tuning, the model can learn domain-specific features and patterns, thus exhibiting higher specialization and accuracy on specific tasks. Large-scale model inference refers to the process of efficiently and accurately processing new data using a trained model to draw conclusions. The inference process does not require retraining the model; instead, it directly uses the trained model for prediction. In specific implementations, a stage identification engine is used to identify the stage of the task data and determine the current stage.

[0062] For example, the stage identification engine module is based on a convolutional neural network (CNN) or random forest algorithm to identify the current stage (training / fine-tuning / inference) through task features (such as gradient update frequency and input sequence length).

[0063] In one optional implementation, after step 102, the method further includes: determining whether the current state is overloaded based on hardware status data; if it is overloaded, then step 104 is executed. For example, if the measured power exceeds a set power, it is determined that the current state is overloaded.

[0064] Step 106: Based on the hardware status data and task data, the prediction model corresponding to the current stage is used to predict the status and obtain the target status information.

[0065] The information to be predicted differs at different stages, and different prediction models are used accordingly. It's understandable that the prediction model for the current stage is trained based on the historical energy consumption logs corresponding to that stage.

[0066] In the specific implementation, the hardware status data and task data are processed into an input sequence that can be directly input into the prediction model. The prediction model predicts the hardware's operating status after a preset time based on this input sequence, thus obtaining the target status information.

[0067] Optionally, the prediction model for the training phase is an energy consumption prediction model, used to predict the target power after a preset time based on hardware status data and task data. The prediction model for the fine-tuning phase is an energy consumption prediction model, used to predict the peak power after a preset time based on hardware status data and task data. The prediction model for the inference phase is a load prediction model, used to predict the request rate after a preset time based on hardware status data and task data.

[0068] Step 108: The joint optimizer generates the optimization parameters for the current stage based on the target state information.

[0069] The joint optimizer generates stage-specific optimization parameters (such as the optimal batch size in the training stage and the number of gradient accumulation steps in the fine-tuning stage) based on the target state information through reinforcement learning (RL).

[0070] Step 110: Adjust the resources of the large model according to the optimization parameters.

[0071] Specifically, based on the optimization parameters, the software or hardware resources for the large model to perform the current stage of the task are adjusted, thereby optimizing the energy consumption of the large model.

[0072] The aforementioned large-scale model energy consumption optimization method involves collecting hardware status data and large-scale model task data; identifying the current stage based on the task data; using the prediction model corresponding to the current stage to predict the target state information based on the hardware status data and task data; generating optimization parameters for the current stage using a joint optimizer based on the target state information; and adjusting the resources of the large-scale model based on the optimization parameters. By combining stage awareness with dynamic resource adjustment, this method achieves differentiated resource adjustment strategies based on different stages, addressing the unique energy consumption challenges in AI scenarios. It reduces the high power consumption of large-scale model training and the instantaneous fluctuations in inference requests, ensuring the sustainable and efficient operation of the AI ​​system, balancing the computing power requirements and resource consumption of large-scale models, and facilitating the widespread deployment and development of large-scale models in various scenarios.

[0073] In an exemplary embodiment, the current stage includes a training stage, where the prediction model is an energy consumption prediction model, the target state information predicted by the energy consumption prediction model is the target power, and the optimization parameters include the optimal batch size and frozen network layer parameters; step 108 includes: determining adjustment coefficients based on the current power and the target power using a joint optimizer; determining the optimal batch size based on the adjustment coefficients and the current batch size; identifying network layers in the large model whose contribution value is less than a set value; and determining frozen network layer parameters based on the identified network layers.

[0074] Accordingly, step 110 includes: adjusting the training parameters of the large model according to the optimal batch size; and freezing the corresponding network layers in the large model according to the frozen network layer parameters.

[0075] Optionally, the energy consumption prediction model is based on a regression tree ensemble algorithm to predict the energy consumption of different hardware performing tasks (such as the energy consumption of GPUs performing inference tasks). In this embodiment, the energy consumption prediction model predicts the energy consumption of hardware performing training tasks based on the current batch size, and then determines the target power.

[0076] Based on the predicted parameters, the following stages of optimization are performed: 1) Dynamic batch processing adjustment: based on the measured current power With the predicted target power Dynamically adjusting the training batch size avoids power overload or resource idleness during the training phase due to a fixed batch size, thereby reducing energy consumption. The specific formula is as follows:

[0077] ;

[0078] In the formula, Indicates the optimal batch size. Indicates the current batch size. This represents the adjustment factor.

[0079] 2) Parameter Freeze Manager: Identifies low-contribution layers (network layers with contribution values ​​less than a set value) in large models through attention weight analysis and freezes their parameter updates, which can effectively reduce the amount of computation.

[0080] It is understandable that the training phase is a computationally intensive task that requires continuous high power support, with hardware utilization approaching 100% for extended periods. To address this issue, this embodiment dynamically adjusts the batch size, which avoids power overload or resource idleness caused by a fixed batch size during the training phase, thereby reducing energy consumption.

[0081] In an exemplary embodiment, the current stage includes a fine-tuning stage, the prediction model of the fine-tuning stage is an energy consumption prediction model, the target state information predicted by the energy consumption prediction model is the power peak, and the optimization parameters include the gradient accumulation step number and the fine-tuning strategy setting parameters; step 108 includes: determining the gradient accumulation step number based on the power peak and the current power using a joint optimizer; and determining the fine-tuning strategy setting parameters using a PEFT parameter selector.

[0082] Accordingly, step 110 includes: adding gradient accumulation steps to the fine-tuning phase of the large model, and adjusting the fine-tuning strategy of the large model according to the parameters set by the fine-tuning strategy.

[0083] The energy consumption prediction model predicts the energy consumption of the hardware during training tasks based on hardware status data and task data, thereby determining the peak power. Based on the predicted parameters, the following optimization stages are performed:

[0084] 1) Gradient Accumulator Scheduler: Automatically increases the gradient accumulation steps during power peaks to smooth out instantaneous power spikes during the fine-tuning phase, thereby reducing energy consumption. The formula for the gradient accumulation steps is as follows:

[0085] ;

[0086] In the formula, the peak power is calculated first. Compared with the measured current power The ratio, since it may be a decimal, needs to be rounded up to obtain an integer gradient accumulation step count. This gradient accumulation step is added to the fine-tuning phase of the large model to enable the allocation of fine-tuning resources.

[0087] 2) PEFT parameter selector: Automatically selects low-energy fine-tuning strategies such as LoRA / QLoRA, sets parameters according to the fine-tuning strategy output by the PEFT parameter selector, and sets the fine-tuning strategy for large models. In this way, energy consumption is reduced by 70% compared to full parameter fine-tuning.

[0088] Understandably, the fine-tuning phase features significant dynamic updates from small samples, with a high parameter update frequency but non-continuous peak computational cost. Traditional strategies, lacking phase awareness, result in instantaneous power spikes. In this embodiment, setting gradient accumulation steps and a fine-tuning strategy during the fine-tuning phase can reduce energy consumption.

[0089] In an exemplary embodiment, the current stage includes an inference stage, the prediction model of the inference stage is a load prediction model, the target state information predicted by the load prediction model is the request rate, and the optimization parameters include target accuracy and request queue configuration parameters; step 108 includes: selecting the target accuracy from multiple preset large model accuracies based on the request rate through a joint optimizer, and determining the request queue configuration parameters.

[0090] Accordingly, step 110 includes: adjusting the accuracy of the large model to the target accuracy; configuring the request processing mechanism of the request queue according to the request queue configuration parameters; the request processing mechanism includes setting priorities for requests in the request queue and allocating computing resources to high-priority requests first.

[0091] The load prediction model adopts a hybrid model of time-varying Poisson process + LSTM (Long Short-Term Memory) to predict the request rate λ(t) during the inference stage based on hardware status data and task data, with a root mean square error (RMSE) ≤ 1.2kW.

[0092] Based on the predicted parameters, the following optimization stages are performed:

[0093] 1) Adaptive Quantization Engine: Dynamically switches accuracy (INT8 / FP16) based on predicted load to balance inference latency and energy consumption; 2) Request Queue Priority Controller: Classifies requests based on queue theory (e.g., M / M / 1 model), sets priorities for requests in the request queue, and allocates GPU resources to high-priority requests first.

[0094] It is understandable that the random load characteristics driven by user behavior during the inference phase, coupled with the fact that existing models do not model the non-stationarity of request arrival times, lead to a high prediction error rate. This embodiment integrates a time-varying Poisson process and an LSTM model for load prediction, simultaneously characterizing the randomness (Poisson noise) and temporal correlation (LSTM) of inference requests. This approach is suitable for non-stationary loads in AI tasks (such as sudden bursts of user queries), effectively improving prediction accuracy.

[0095] In an exemplary embodiment, after step 110, the method further includes: assigning the task of the current stage to the corresponding optimal hardware for processing based on the Hungarian algorithm; and dynamically adjusting the voltage or frequency of the hardware according to the task execution status.

[0096] Among them, the optimal matching of tasks and hardware is achieved based on the Hungarian algorithm, such as assigning matrix operations to the GPU and logic control to the CPU, with a scheduling latency of ≤100μs; through the dynamic voltage-frequency adjustment (DVFS) technology, the voltage or frequency of the hardware is dynamically adjusted according to the task execution status, reducing hardware power consumption during task intervals, such as the GPU voltage dropping from 1.3V to 0.8V when idle.

[0097] In this embodiment, a heterogeneous scheduler is used to match the tasks and energy efficiency of hardware such as GPU / TPU / CPU. At the same time, the power controller (DVFS) is used to dynamically adjust the hardware voltage or frequency, forming a software and hardware collaboration from algorithm strategy to hardware execution, thereby achieving energy consumption optimization during the operation of large models.

[0098] It is understood that this application's embodiments address the significant differences in resource requirements during the training, fine-tuning, and inference stages of large-scale model operations by constructing an energy consumption optimization system based on precise perception of the operation stages, dynamic modeling of random loads, and hardware collaborative optimization. Specifically, high-precision identification of each stage (training / fine-tuning / inference) is achieved through the CNN+Random Forest algorithm, adapting to differentiated strategies such as dynamic batch processing, gradient accumulation scheduling, and adaptive quantization, thereby optimizing the energy consumption of each stage of the large model. A hybrid model integrating time-varying Poisson and LSTM is used to reduce the inference load prediction error to RMSE≤1.2kW, accurately capturing millisecond-level power fluctuations. The Hungarian algorithm is used in conjunction for heterogeneous hardware scheduling, matching tasks with hardware energy efficiency, effectively improving the energy efficiency ratio of the AI ​​system. While ensuring training convergence speed and inference service quality (e.g., latency exceedance rate <5%), it solves the problems of convergence delay, instantaneous power spikes, and high prediction error rates caused by the "one-size-fits-all" approach of existing solutions.

[0099] In an exemplary embodiment, the method further includes: collecting measured state information; determining the prediction error based on the measured state information and the target state information; and training and updating the prediction model corresponding to the current stage if the prediction error is greater than a preset value.

[0100] Among them, power data is collected through a current sensor and an analog-to-digital converter (ADC) with an accuracy of ≤±1%; the predicted value is compared with the measured value to trigger the model parameter update; if the prediction error is greater than a preset value, the prediction model corresponding to the current stage is trained and updated (e.g., online learning is started when RMSE>5%).

[0101] For example, the method further includes: using a task result evaluation module to monitor metrics such as inference latency and training convergence speed, and feeding these metrics back to the joint optimizer for policy adjustment. For instance, adjusting the reinforcement learning parameters of the joint optimizer.

[0102] It is understood that the embodiments of this application cover the entire chain of "input → recognition → optimization → execution → feedback". Through dynamic stage recognition, hybrid model prediction, cross-hardware collaborative optimization, and real-time closed-loop correction, it realizes stage-specific energy efficiency management of AI tasks, which can improve training convergence speed, reduce inference latency, and improve hardware utilization, thus solving the contradiction between energy efficiency and performance requirements in AI heterogeneous computing scenarios.

[0103] In one exemplary embodiment, refer to Figure 2 The system used in the large-model energy consumption optimization method provided in this application embodiment includes: an input layer, a core control layer, a stage optimization layer, a hardware execution layer, a feedback layer, and a hardware layer. Wherein:

[0104] 1. The input layer includes a task data / user request module, a status acquisition module, and a historical energy consumption log module. The task data / user request module receives large model training data, fine-tuning instructions, or inference requests, including features such as sequence length and batch size; the status acquisition module receives real-time data from hardware sensors in the hardware layer, such as GPU temperature, memory bandwidth, and CPU load (sampling frequency ≥ 1kHz); the historical energy consumption log module stores energy consumption data from past tasks, which is used to train load prediction models or energy consumption prediction models.

[0105] 2. The stage strategy library module of the core control layer receives and analyzes data from the stage identification engine, load prediction model, energy consumption prediction model, and joint optimizer. The stage identification engine module, based on a Convolutional Neural Network (CNN) / Random Forest algorithm, identifies the current stage (training / fine-tuning / inference) based on task characteristics (such as gradient update frequency and input sequence length). The random load prediction model module uses a time-varying Poisson process + LSTM hybrid model to predict the request rate during the inference stage, with a root mean square error (RMSE) ≤ 1.2kW. The energy consumption prediction model module, based on a regression tree ensemble algorithm, predicts the energy consumption of different hardware for performing tasks (such as the energy consumption of GPUs performing inference tasks, with a prediction error ≤ 5%). The joint optimizer module generates stage-specific optimization parameters (such as the optimal batch size during training and the number of gradient accumulation steps during fine-tuning) through reinforcement learning (RL).

[0106] 3. In the stage optimization layer, the following stage optimizations are performed for the training stage: 1) Dynamic batch processing adjustment: based on the measured current power With the predicted target power Dynamically adjusting the training batch size (BatchSize) avoids power overload or resource idleness during the training phase caused by a fixed batch size, thereby reducing energy consumption. The specific formula is as follows:

[0107] ;

[0108] In the formula, Indicates the optimal batch size. Indicates the current batch size. This represents the adjustment factor.

[0109] 2) Parameter Freeze Manager: Identifies low-contribution layers (network layers with contribution values ​​less than a set value) in large models through attention weight analysis and freezes their parameter updates, which can effectively reduce the amount of computation.

[0110] For the fine-tuning phase, the following optimizations are performed: 1) Gradient accumulation scheduler: Automatically increase the gradient accumulation steps during power peaks to smooth out instantaneous power spikes during the fine-tuning phase, thereby reducing energy consumption. The formula for the gradient accumulation steps is as follows:

[0111] ;

[0112] In the formula, the peak power is calculated first. Compared with the measured current power The ratio, since it may be a decimal, needs to be rounded up to obtain an integer gradient accumulation step count. This gradient accumulation step is added to the fine-tuning phase of the large model to enable the allocation of fine-tuning resources.

[0113] 2) PEFT parameter selector: Automatically selects low-energy fine-tuning strategies such as LoRA / QLoRA, sets parameters according to the fine-tuning strategy output by the PEFT parameter selector, and sets the fine-tuning strategy for large models. In this way, energy consumption is reduced by 70% compared to full parameter fine-tuning.

[0114] For the inference phase, the following optimizations are performed: 1) Adaptive quantization engine: dynamically switch the accuracy (INT8 / FP16) according to the prediction load to balance inference latency and energy consumption; 2) Request queue priority controller: classify requests based on queue theory (e.g., M / M / 1 model), set priorities for requests in the request queue, and allocate GPU resources to high-priority requests first.

[0115] 4. In the hardware execution layer, the heterogeneous scheduler achieves optimal matching between tasks and hardware based on the Hungarian algorithm (e.g., assigning matrix operations to the GPU and logic control to the CPU), with a scheduling latency of ≤100μs; the power controller reduces hardware power consumption during task gaps through Dynamic Voltage Frequency Scaling (DVFS) technology (e.g., the GPU voltage drops from 1.3V to 0.8V when idle).

[0116] 5. In the feedback closed-loop layer, the real-time power monitoring module collects power data through a current sensor and an analog-to-digital converter (ADC) with an accuracy of ≤±1%; the error analyzer module compares the predicted value with the measured value and triggers the model parameter update (e.g., online learning is started when RMSE>5%); the task result evaluation module monitors inference latency, training convergence speed and other indicators and feeds them back to the joint optimizer for strategy adjustment.

[0117] In one exemplary embodiment, refer to Figure 3 The energy consumption optimization method for large models includes the following steps:

[0118] The first step is data acquisition: Hardware sensors collect real-time status data such as temperature, voltage, and memory bandwidth of processors such as GPU, TPU, and CPU (sampling frequency ≥ 1kHz). The task data, including training samples and user inference requests, is uploaded to the input layer. Features such as batch size, sequence length, and gradient update frequency are extracted, and historical energy consumption logs are loaded for training the prediction model.

[0119] It is understandable that the data uploaded by the input layer, the output λ(t) of the load prediction model, and the hardware energy efficiency data of the energy consumption prediction model jointly determine the optimization parameters of each stage in the stage strategy library (such as the quantization accuracy in the inference stage and the optimal batch size in the training stage).

[0120] The second step is stage identification: The stage identification engine, based on CNN or random forest algorithms, analyzes task features and determines the current operation stage (training / fine-tuning / inference). It outputs a stage label (e.g., "training") with a confidence level ≥ 95%.

[0121] The third step is prediction and strategy generation: The stage strategy library issues specific optimization instructions to the dynamic batch processing, gradient accumulation scheduling, and adaptive quantization engine based on the current operation stage (training / fine-tuning / inference). When the system is identified as being in the training stage, the energy consumption prediction model calculates the theoretical power consumption under the current batch size, and the dynamic batch processing module adjusts parameters according to the formulas mentioned above; it also automatically identifies layers with low energy consumption contribution (such as early convolutional layers) and dynamically freezes parameter updates. When the system is identified as being in the fine-tuning stage, the gradient accumulation scheduler determines the accumulation steps based on the power peak, and the PEFT selector automatically switches to low-energy fine-tuning strategies such as LoRA. When the system is identified as being in the inference stage, the stochastic load prediction model (time-varying Poisson + LSTM) predicts the request rate λ(t) within the next 100ms; the adaptive quantization engine dynamically selects INT8 / FP16 precision based on the load.

[0122] The fourth step is hardware scheduling and execution: The heterogeneous scheduler, based on the Hungarian algorithm, allocates tasks to the optimal hardware (e.g., matrix operations → GPU, logic control → CPU); the power controller adjusts the hardware voltage / frequency in real time (e.g., the GPU voltage drops from 1.3V to 0.8V when idle); the execution unit (GPU / TPU / CPU) completes the calculation and outputs the results.

[0123] Step 5, Feedback Optimization: The power monitoring module collects measured power data at a frequency of 1kHz; the error analyzer updates the prediction model parameters every 100ms and compares the predicted values ​​with the measured values. When RMSE>5%, the model parameters are updated; the result evaluation module monitors inference latency, training convergence speed and other indicators, and feeds them back to the joint optimizer to adjust the request priority strategy.

[0124] For example, in an AI inference cluster of a cloud computing data center, there are 100 GPU servers that process real-time user query requests. The load prediction model (time-varying Poisson + LSTM) initially predicts a request rate λ(t) of 50 times / second during the inference phase, but in reality, a sudden surge in user access causes the request rate to spike to 80 times / second, triggering power fluctuations.

[0125] The data transfer steps are as follows:

[0126] 1) Real-time power acquisition: The power monitoring module acquires the total power of the GPU cluster at a frequency of 1kHz and finds that the measured value suddenly increased from 200kW to 280kW, exceeding the predicted value (220kW) by 30%.

[0127] Monitoring data format: {timestamp:year-month-day hour:minute:second, measured power:280kW, predicted power:220kW, error rate:27.3%}.

[0128] 2) Error Analysis and Model Triggering: The error analyzer calculates RMSE = 27.3%, exceeding the threshold of 5%, and sends a correction signal to the stochastic load prediction model. Triggering condition: if (measured power - predicted power) / predicted power > 20%, then online learning is initiated.

[0129] 3) Prediction model parameter update: The random load prediction model receives error data, extracts features of the current time period (such as user access patterns at 3 pm on weekdays), and retrains through the LSTM network.

[0130] The training process includes: inputting the following features: the request sequence of the past hour (including time features, such as hour / minute encoding) and measured power values; outputting the following features: updating the time-varying parameter λ(t) of the Poisson process and introducing the weight of recent burst loads (e.g., assigning 40% weight to the data of the last 10 minutes).

[0131] Formula update: λnew(t) = λold(t) × (1 + 0.5 × error rate);

[0132] In this example, λ(t) is adjusted from 50 times / second to 63.6 times / second.

[0133] 4) Dynamic strategy adjustment: Based on the new λ(t) = 63.6 times / second, the adaptive quantization engine switches the model accuracy from FP16 to INT8, reducing the computational load by 20%, the measured power drops back to 250kW, and the error rate drops to 12%.

[0134] For example, an AI lab trained a large language model (1.8 trillion parameters) on the scale of GPT-4 using eight 8-GPU servers. The initial batch size was set to 128, and the predicted target power P_target = 500kW. During training, the measured power P_current surged to 650kW, exceeding the target power by 30%, triggering overload protection.

[0135] The adjustment process steps are as follows:

[0136] 1) Overload detection and stage identification: The stage identification engine confirms that the current stage is the training stage (batch size > 128 and gradient update frequency > 0.9); the power monitoring module records: P_current = 650kW, P_target = 500kW, overload rate = 30%.

[0137] 2) Dynamic batch processing formula calculation:

[0138] ;

[0139] Batch size is inversely proportional to the square root of power; reducing batch size reduces computational load, thereby reducing power consumption.

[0140] 3) Parameter freezing collaborative optimization: synchronously trigger parameter freezing technology: analyze the attention weights of each layer of the model, freeze the parameters of the first 3 convolutional layers with a contribution of <10%, and reduce the amount of computation by 15%.

[0141] After freezing, the measured power dropped to 600kW, which is still 20% higher than the target value, so a second adjustment is needed.

[0142] 4) Secondary adjustment and feedback verification: Calculate again using the formula:

[0143] ;

[0144] During the second adjustment, the batch size was changed to 100, and the actual power dropped to 520kW, which is close to the target value (error 4%), so the adjustment was stopped.

[0145] 5) Training progress and accuracy verification: After adjustment, the training convergence speed decreased by only 5% (compensated by gradient accumulation), and the model accuracy loss was <1% (the perplexity of the validation set increased from 10.2 to 10.3).

[0146] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0147] Based on the same inventive concept, this application also provides a large model energy consumption optimization device for implementing the large model energy consumption optimization method described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more large model energy consumption optimization device embodiments provided below can be found in the limitations of the large model energy consumption optimization method above, and will not be repeated here.

[0148] In one exemplary embodiment, such as Figure 4 As shown, a large-scale model energy consumption optimization device is provided, comprising:

[0149] Input module 402 is used to collect hardware status data and task data of large models.

[0150] The stage identification module 404 is used to identify the current stage based on the task data.

[0151] The state prediction module 406 is used to predict the state based on hardware state data and task data using the prediction model corresponding to the current stage, and obtain the target state information.

[0152] The optimization analysis module 408 is used to generate optimization parameters for the current stage based on the target state information through a joint optimizer.

[0153] The stage optimization module 410 is used to adjust the resources of the large model according to the optimization parameters.

[0154] The aforementioned large-scale model energy consumption optimization device collects hardware status data and large-scale model task data; it identifies the current stage based on the task data; it then uses the prediction model corresponding to the current stage to predict the target state information based on the hardware status data and task data; a joint optimizer generates optimization parameters for the current stage based on the target state information; and finally, it adjusts the resources of the large-scale model according to the optimization parameters. By combining stage awareness with dynamic resource adjustment, this device implements differentiated resource adjustment strategies based on different stages, addressing the unique energy consumption challenges in AI scenarios. It reduces the high power consumption of large-scale model training and the instantaneous fluctuations in inference requests, ensuring the sustainable and efficient operation of the AI ​​system, balancing the computing power requirements and resource consumption of large-scale models, and contributing to the widespread deployment and development of large-scale models in various scenarios.

[0155] In an exemplary embodiment, the current stage includes a training stage, where the prediction model is an energy consumption prediction model, the target state information predicted by the energy consumption prediction model is the target power, and the optimization parameters include the optimal batch size and the frozen network layer parameters. The optimization analysis module 408 is further configured to determine the adjustment coefficients based on the current power and the target power using a joint optimizer; determine the optimal batch size based on the adjustment coefficients and the current batch size; identify network layers in the large model whose contribution values ​​are less than a set value; and determine the frozen network layer parameters based on the identified network layers.

[0156] Correspondingly, the stage optimization module 410 is also used to adjust the training parameters of the large model according to the optimal batch size; and to freeze the corresponding network layers in the large model according to the frozen network layer parameters.

[0157] In an exemplary embodiment, the current stage includes a fine-tuning stage, the prediction model of the fine-tuning stage is an energy consumption prediction model, the target state information predicted by the energy consumption prediction model is the power peak, and the optimization parameters include the gradient accumulation step number and the fine-tuning strategy setting parameters; the optimization analysis module 408 is further used to determine the gradient accumulation step number based on the power peak and the current power through a joint optimizer; and to determine the fine-tuning strategy setting parameters using a PEFT parameter selector.

[0158] Correspondingly, the stage optimization module 410 is also used to add gradient accumulation steps to the fine-tuning stage of the large model, and to adjust the fine-tuning strategy of the large model according to the parameters set by the fine-tuning strategy.

[0159] In an exemplary embodiment, the current stage includes an inference stage, the prediction model of the inference stage is a load prediction model, the target state information predicted by the load prediction model is the request rate, and the optimization parameters include target accuracy and request queue configuration parameters; the optimization analysis module 408 is also used to select the target accuracy from multiple preset large model accuracies based on the request rate through a joint optimizer, and determine the request queue configuration parameters.

[0160] Correspondingly, the stage optimization module 410 is also used to adjust the accuracy of the large model to the target accuracy; and to configure the request processing mechanism of the request queue according to the request queue configuration parameters; the request processing mechanism includes setting priorities for requests in the request queue and allocating computing resources to high-priority requests first.

[0161] In an exemplary embodiment, the large model energy consumption optimization device further includes a hardware execution module; the hardware execution module is used to allocate the tasks of the current stage to the corresponding optimal hardware for processing based on the Hungarian algorithm; and to dynamically adjust the voltage or frequency of the hardware according to the task execution status.

[0162] In an exemplary embodiment, the large model energy consumption optimization device further includes a feedback module; the feedback module is used to collect measured state information; determine the prediction error based on the measured state information and the target state information; and train and update the prediction model corresponding to the current stage when the prediction error is greater than a preset value.

[0163] Each module in the aforementioned large-scale energy consumption optimization device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0164] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data generated during large-scale model processing. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a large-scale model energy consumption optimization method.

[0165] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0166] In one exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: acquiring hardware status data and task data of a large model; identifying stages based on the task data to determine the current stage; predicting the state based on the hardware status data and task data using a prediction model corresponding to the current stage to obtain target state information; generating optimization parameters for the current stage based on the target state information using a joint optimizer; and adjusting the resources of the large model based on the optimization parameters.

[0167] In one embodiment, when the processor executes the computer program, it further performs the following steps: determining adjustment coefficients based on the current power and the target power using a joint optimizer; determining the optimal batch size based on the adjustment coefficients and the current batch size; identifying network layers in the large model whose contribution values ​​are less than a set value; determining the parameters to freeze network layers based on the identified network layers; adjusting the training parameters of the large model based on the optimal batch size; and freezing the corresponding network layers in the large model based on the parameters to freeze network layers.

[0168] In one embodiment, when the processor executes the computer program, it also performs the following steps: determining the gradient accumulation steps based on the power peak and the current power using a joint optimizer; determining the fine-tuning strategy setting parameters using a PEFT parameter selector; adding the gradient accumulation steps to the fine-tuning phase of the large model; and adjusting the fine-tuning strategy of the large model according to the fine-tuning strategy setting parameters.

[0169] In one embodiment, when the processor executes the computer program, it further performs the following steps: selecting a target precision from multiple preset large model precisions based on the request rate using a joint optimizer, and determining request queue configuration parameters; adjusting the precision of the large model to the target precision; configuring the request processing mechanism of the request queue according to the request queue configuration parameters; the request processing mechanism includes setting priorities for requests in the request queue and allocating computing resources to high-priority requests first.

[0170] In one embodiment, when the processor executes the computer program, it also performs the following steps: based on the Hungarian algorithm, assigning the task of the current stage to the corresponding optimal hardware for processing; and dynamically adjusting the voltage or frequency of the hardware according to the task execution status.

[0171] In one embodiment, when the processor executes the computer program, it also performs the following steps: collecting measured state information; determining the prediction error based on the measured state information and the target state information; and training and updating the prediction model corresponding to the current stage if the prediction error is greater than a preset value.

[0172] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps: collecting hardware status data and task data of a large model; identifying stages based on the task data to determine the current stage; predicting the state based on the hardware status data and task data using a prediction model corresponding to the current stage to obtain target state information; generating optimization parameters for the current stage based on the target state information using a joint optimizer; and adjusting the resources of the large model based on the optimization parameters.

[0173] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining adjustment coefficients based on the current power and the target power using a joint optimizer; determining the optimal batch size based on the adjustment coefficients and the current batch size; identifying network layers in the large model whose contribution values ​​are less than a set value; determining the parameters to freeze network layers based on the identified network layers; adjusting the training parameters of the large model based on the optimal batch size; and freezing the corresponding network layers in the large model based on the parameters to freeze network layers.

[0174] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the gradient accumulation steps based on the power peak and the current power using a joint optimizer; determining the fine-tuning strategy setting parameters using a PEFT parameter selector; adding the gradient accumulation steps to the fine-tuning phase of the large model; and adjusting the fine-tuning strategy of the large model according to the fine-tuning strategy setting parameters.

[0175] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: selecting a target precision from multiple preset large model precisions based on the request rate using a joint optimizer, and determining request queue configuration parameters; adjusting the precision of the large model to the target precision; configuring the request processing mechanism of the request queue according to the request queue configuration parameters; the request processing mechanism includes setting priorities for requests in the request queue and allocating computing resources to high-priority requests first.

[0176] In one embodiment, when the computer program is executed by the processor, it also performs the following steps: based on the Hungarian algorithm, assigning the task of the current stage to the corresponding optimal hardware for processing; and dynamically adjusting the voltage or frequency of the hardware according to the task execution status.

[0177] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: collecting measured state information; determining the prediction error based on the measured state information and the target state information; and training and updating the prediction model corresponding to the current stage if the prediction error is greater than a preset value.

[0178] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps: collecting hardware status data and task data of a large model; identifying stages based on the task data to determine the current stage; predicting the state based on the hardware status data and task data using a prediction model corresponding to the current stage to obtain target state information; generating optimization parameters for the current stage based on the target state information using a joint optimizer; and adjusting the resources of the large model based on the optimization parameters.

[0179] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining adjustment coefficients based on the current power and the target power using a joint optimizer; determining the optimal batch size based on the adjustment coefficients and the current batch size; identifying network layers in the large model whose contribution values ​​are less than a set value; determining the parameters to freeze network layers based on the identified network layers; adjusting the training parameters of the large model based on the optimal batch size; and freezing the corresponding network layers in the large model based on the parameters to freeze network layers.

[0180] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: determining the gradient accumulation steps based on the power peak and the current power using a joint optimizer; determining the fine-tuning strategy setting parameters using a PEFT parameter selector; adding the gradient accumulation steps to the fine-tuning phase of the large model; and adjusting the fine-tuning strategy of the large model according to the fine-tuning strategy setting parameters.

[0181] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: selecting a target precision from multiple preset large model precisions based on the request rate using a joint optimizer, and determining request queue configuration parameters; adjusting the precision of the large model to the target precision; configuring the request processing mechanism of the request queue according to the request queue configuration parameters; the request processing mechanism includes setting priorities for requests in the request queue and allocating computing resources to high-priority requests first.

[0182] In one embodiment, when the computer program is executed by the processor, it also performs the following steps: based on the Hungarian algorithm, assigning the task of the current stage to the corresponding optimal hardware for processing; and dynamically adjusting the voltage or frequency of the hardware according to the task execution status.

[0183] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: collecting measured state information; determining the prediction error based on the measured state information and the target state information; and training and updating the prediction model corresponding to the current stage if the prediction error is greater than a preset value.

[0184] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0185] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0186] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0187] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for optimizing energy consumption in a large-scale model, characterized in that, The method includes: Collect hardware status data and task data from large models; Based on the task data, stage identification is performed to determine the current stage; The target state information is obtained by using the prediction model corresponding to the current stage to predict the state based on the hardware state data and the task data. The joint optimizer generates the optimization parameters for the current stage based on the target state information. Based on the optimization parameters, the resources of the large model are adjusted.

2. The method according to claim 1, characterized in that, The current stage includes a training stage, where the prediction model is an energy consumption prediction model, and the target state information predicted by the energy consumption prediction model is the target power. The optimization parameters include the optimal batch size and the parameters for freezing network layers. The generation of the optimization parameters for the current stage using a joint optimizer based on the target state information includes: The adjustment coefficients are determined by a joint optimizer based on the current power and the target power; Determine the optimal batch size based on the adjustment coefficient and the current batch size; Identify network layers in the large model whose contribution value is less than a set value; Based on the identified network layers, determine the parameters for freezing the network layers; Accordingly, adjusting the resources of the large model based on the optimization parameters includes: The training parameters of the large model are adjusted according to the optimal batch size. Based on the parameters of the frozen network layers, the corresponding network layers in the large model are frozen.

3. The method according to claim 1, characterized in that, The current stage includes a fine-tuning stage, the prediction model of the fine-tuning stage is an energy consumption prediction model, the target state information predicted by the energy consumption prediction model is the power peak, and the optimization parameters include gradient accumulation steps and fine-tuning strategy setting parameters. The step of generating optimization parameters for the current stage using a joint optimizer based on the target state information includes: The joint optimizer determines the number of gradient accumulation steps based on the peak power and the current power. Use the PEFT parameter selector to determine the fine-tuning strategy settings parameters; Accordingly, adjusting the resources of the large model based on the optimization parameters includes: The gradient accumulation steps are added to the fine-tuning phase of the large model, and the fine-tuning strategy of the large model is adjusted according to the parameters set by the fine-tuning strategy.

4. The method according to claim 1, characterized in that, The current stage includes an inference stage, where the prediction model is a load prediction model, and the target state information predicted by the load prediction model is the request rate. The optimization parameters include target accuracy and request queue configuration parameters. The step of generating the optimization parameters for the current stage using a joint optimizer based on the target state information includes: The joint optimizer selects the target accuracy from multiple preset large model accuracies based on the request rate and determines the request queue configuration parameters. Accordingly, adjusting the resources of the large model based on the optimization parameters includes: Adjust the accuracy of the large model to the target accuracy; Configure the request processing mechanism of the request queue according to the request queue configuration parameters; the request processing mechanism includes setting priorities for requests in the request queue and allocating computing resources to high-priority requests first.

5. The method according to any one of claims 1 to 4, characterized in that, After adjusting the resources of the large model according to the optimization parameters, the method further includes: Based on the Hungarian algorithm, the tasks of the current stage are assigned to the corresponding optimal hardware for processing; The hardware voltage or frequency is dynamically adjusted based on the task execution status.

6. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Collect measured status information; The prediction error is determined based on the measured state information and the target state information; If the prediction error is greater than a preset value, the prediction model corresponding to the current stage is trained and updated.

7. A large-scale model energy consumption optimization device, characterized in that, The device includes: The input module is used to collect hardware status data and task data for large models; The stage identification module is used to identify the current stage based on the task data. The state prediction module is used to predict the state based on the hardware state data and the task data using the prediction model corresponding to the current stage, and obtain the target state information. The optimization analysis module is used to generate the optimization parameters for the current stage based on the target state information through a joint optimizer; The phase optimization module is used to adjust the resources of the large model according to the optimization parameters.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images