Two-stage large language model energy consumption analysis method and system for inference task

Abstract

The application discloses a two-stage large language model energy consumption analysis method and system for reasoning tasks, and belongs to the technical field of data mining.The application effectively solves the modeling problem of key dynamic characteristics by adding a response token number interval prediction stage, so that the energy consumption model can indirectly use output length information, and innovatively constructs and fuses enhanced features such as model total parameter count and theoretical floating point operation number, which have clear physical meanings.The system integrates multi-dimensional heterogeneous information such as prompt words, models and hardware, and realizes fine description of energy consumption differences of different reasoning tasks.The proposed two-stage framework of classification prediction and regression modeling has clear logic and is consistent with the energy consumption characteristics of the LLM reasoning process.The first stage focuses on predicting features strongly related to output, and the second stage performs comprehensive energy consumption regression, so that the model structure is reasonable and has strong interpretability.

Description

Technical Field

[0001] This invention belongs to the technical field of artificial intelligence, specifically relating to a two-stage energy consumption analysis method and system for large-scale language models oriented towards reasoning tasks. Background Technology

[0002] With the rise of Large Language Models (LLMs), the frequency and scale of inference requests continue to grow. To support the workload of LLM inference tasks, the energy consumption of intelligent computing data centers continues to rise. For example, OpenAI spends approximately $700,000 per day just running ChatGPT, and maintaining its daily operations consumes a significant amount of electricity and computing resources. In March 2024, ChatGPT processed over 200 million requests per day, with its electricity consumption potentially reaching 500,000 kilowatt-hours per day—more than 17,000 times the daily electricity consumption of a typical household. This surge in energy demand not only places a heavy burden on the power grid but also increases the operating costs of data centers and exacerbates environmental problems through carbon emissions. Therefore, building accurate and systematic energy consumption models has become an important measure to improve data center efficiency, support power grid stability, and promote the sustainable development of artificial intelligence.

[0003] Several studies have already modeled the energy consumption of data centers and inference tasks:

[0004] 1. CN120704868A A method for optimizing the scheduling of energy consumption for large language model inference computing services based on task length prediction: This paper discloses a method for optimizing the scheduling of energy consumption for large language model inference computing services based on task length prediction, so as to reduce the energy consumption of large language model inference computing.

[0005] 2. CN117331699A An edge-to-edge collaborative AI model inference method based on energy consumption prediction: It proposes a method to integrate several AI models into an AI fusion model, and divide the AI ​​fusion model into n type layers. Based on the feature parameters of different type layers of the AI ​​fusion model, the latency and energy consumption of each type layer are obtained for inference.

[0006] 3. CN120671490A A method for short-term prediction of data center cooling energy consumption based on feature selection-fusion and deep learning: A short-term prediction model for cooling energy consumption based on SEN-MH-BiLSTM is proposed, which improves the accuracy of cooling energy consumption prediction.

[0007] These studies provide valuable insights into data center energy consumption modeling, but fail to capture the dynamic characteristics and heterogeneity of large language model inference tasks, thus limiting their direct application to energy consumption modeling for large language model inference. Unlike traditional system-level data center energy consumption modeling, which typically considers both computational and non-computational energy consumption such as cooling, this research focuses on energy consumption analysis at the inference task level. This task-level energy consumption is significantly affected by factors such as input prompt length, model parameters, and heterogeneous hardware resource configurations, factors that traditional models do not explicitly consider. The inference process is complex, comprising two stages: pre-filling and decoding. These two stages differ fundamentally in computational patterns, resource bottlenecks, and energy consumption patterns. The pre-filling stage processes the entire input sequence in parallel, requiring intensive computation in a short time. The decoding stage generates tokens sequentially, requiring less computation at each step but iterative execution until a response is completed. These different computational patterns lead to differences in energy consumption characteristics during the inference process. Traditional modeling methods cannot accurately capture these differences in energy consumption patterns, thus failing to provide precise energy consumption modeling. Summary of the Invention

[0008] To address the shortcomings of the existing technologies, this application provides a two-stage energy consumption analysis method and system for large-scale language models oriented towards reasoning tasks.

[0009] In its first aspect, this application proposes a two-stage energy consumption analysis method for large-scale language models oriented towards reasoning tasks, comprising the following steps:

[0010] Obtain initial known information about the inference task to be predicted, and extract and construct a multidimensional heterogeneous feature set; the multidimensional heterogeneous feature set includes: cue word length D, cue word complexity C, total model parameter count P, theoretical number of floating-point operations per forward propagation IC, and hardware configuration identifier H; wherein, the total model parameter count P is calculated based on the model architecture parameters. Where l is the number of model layers, h is the hidden size, and V is the vocabulary size, the theoretical number of floating-point operations IC is calculated according to the formula. Calculated;

[0011] The multidimensional heterogeneous feature set is input into the first prediction model, which is a classifier trained based on historical task data. The classifier is used to map the reasoning task to a predefined interval category of response token count. The interval category is a discrete category divided by unsupervised clustering analysis of the actual response token count of historical tasks.

[0012] The obtained interval categories are encoded to generate interval category feature vectors; the interval category feature vectors are fused with the multidimensional heterogeneous feature set to form an enhanced feature set; the enhanced feature set is input into the second prediction model to output the energy consumption prediction value of the inference task; the second prediction model is a regression model trained based on the enhanced historical task data, which uses a gradient boosting decision tree as its core to model the complex nonlinear relationship between the enhanced features and energy consumption.

[0013] In some embodiments, the complexity C of the prompt word is quantified by statistically analyzing the proportion of words of a specific part of speech in the input prompt text, wherein the specific part of speech includes at least adjectives and adverbs.

[0014] In some embodiments, the predefined range of response token number categories is determined using the K-means clustering algorithm, and the optimal number of categories is determined by evaluating the silhouette coefficient method.

[0015] In some embodiments, the first prediction model is a multilayer perceptron neural network classifier, whose output layer uses the Softmax activation function, corresponding to each category of the interval category.

[0016] In some embodiments, the interval category feature vector is generated using one-hot encoding.

[0017] In some embodiments, the second prediction model is an XGBoost regression model, whose objective function is a squared loss function including a regularization term, which constrains the number and weight of the leaf nodes of the decision tree.

[0018] In some embodiments, a model training step is also included:

[0019] S0a: Collect historical reasoning task datasets, use the K-means algorithm to cluster the number of their true response tokens, forming K intervals as interval labels;

[0020] S0b: Using the multidimensional heterogeneous feature set of the historical task as input and the interval label obtained in step S0a as the target, train a neural network classifier as the first prediction model.

[0021] S0c: Use the trained first prediction model to perform forward propagation on historical task data to obtain its predicted interval category, encode the interval category and fuse it with the original feature set, and form a sample pair with the real energy consumption value;

[0022] S0d: Using the enhanced training set constructed in step S0c, train an XGBoost regression model to obtain the second prediction model.

[0023] Secondly, this application proposes a two-stage large-scale language model energy consumption analysis system for reasoning tasks, including: a feature engineering module, a first prediction module, a second prediction module, and a data bus and processing logic module;

[0024] The feature engineering module is used to acquire initial known information about the inference task to be predicted, and to extract and construct a multidimensional heterogeneous feature set X. The multidimensional heterogeneous feature set includes: cue word length D, cue word complexity C, total model parameter count P, theoretical number of floating-point operations per forward propagation IC, and hardware configuration identifier H. The total model parameter count P is calculated based on the model architecture parameters. Where l is the number of model layers, h is the hidden size, and V is the vocabulary size, the theoretical number of floating-point operations IC is calculated according to the formula. Calculated;

[0025] The first prediction module is used to input the multidimensional heterogeneous feature set into the first prediction model. The first prediction model is a classifier trained based on historical task data. It is used to map the reasoning task to a predefined interval category of response token count. The interval category is a discrete category divided by unsupervised clustering analysis of the actual response token count of historical tasks.

[0026] The second prediction module is used to encode the obtained interval categories to generate interval category feature vectors; to fuse the interval category feature vectors with the multidimensional heterogeneous feature set to form an enhanced feature set; to input the enhanced feature set into the second prediction model and output the energy consumption prediction value of the inference task; the second prediction model is a regression model trained based on the enhanced historical task data, which uses a gradient boosting decision tree as its core to model the complex nonlinear relationship between the enhanced features and energy consumption;

[0027] The data bus and processing logic module are used to transmit the output of the feature engineering module to the first prediction module, encode the output of the first prediction module and fuse it with the original features, and then transmit it to the second prediction module.

[0028] Thirdly, this application proposes an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0029] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0030] The beneficial effects of this invention are:

[0031] By introducing response token number prediction as an intermediate stage, the problem of "unpredictable" key dynamic features is effectively solved, enabling the final energy consumption model to indirectly utilize output length information, thereby significantly improving prediction accuracy. Experiments show that compared to the single-stage model, this method improves the MAE index by approximately 89% and the R² index by approximately 79%. It systematically integrates multi-dimensional heterogeneous features such as prompt words, models, and hardware, and constructs deep features such as parameter counts and theoretical FLOPs, enabling the model to more accurately characterize energy consumption differences under different tasks and configurations. The two-stage modeling framework naturally reflects the strong dependence of energy consumption on output length. The first stage focuses on feature prediction strongly correlated with output length, while the second stage integrates all information for energy consumption regression. The model structure is reasonable and highly interpretable. This solution can be integrated into intelligent computing data center management systems to achieve rapid and accurate energy consumption prediction for LLM inference tasks that are about to be executed or are currently queued. This provides key data support for implementing refined energy consumption-aware scheduling, elastic resource supply, and green operation, helping to reduce the overall energy consumption and operating costs of data centers. Attached Figure Description

[0032] Figure 1 This is the overall flowchart of the present invention.

[0033] Figure 2 This is a diagram illustrating the comparison of clustering results.

[0034] Figure 3 This is a comparison chart of the clustering distribution results.

[0035] Figure 4 This is a schematic diagram of the two-stage modeling.

[0036] Figure 5 A comparison chart showing the accuracy results of various modeling methods.

[0037] Figure 6 This is a system principle block diagram of the present invention. Detailed Implementation

[0038] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein; rather, these embodiments are provided so that a more thorough understanding of the invention can be achieved and that the full scope of the invention can be conveyed to those skilled in the art.

[0039] Firstly, this application proposes a two-stage energy consumption analysis method for large-scale language models oriented towards reasoning tasks, such as... Figure 1 As shown, it includes the following steps:

[0040] S100: Obtain the initial known information of the inference task to be predicted, and extract and construct a multi-dimensional heterogeneous feature set; the multi-dimensional heterogeneous feature set includes: prompt word length D, prompt word complexity C, total model parameter count P, theoretical number of floating-point operations per forward propagation IC, and hardware configuration identifier H; wherein, the total model parameter count P is calculated based on the model architecture parameters: Where l is the number of model layers, h is the hidden size, and V is the vocabulary size, the theoretical number of floating-point operations IC is calculated according to the formula. Calculated;

[0041] In some embodiments, the complexity C of the prompt word is quantified by statistically analyzing the proportion of words of a specific part of speech in the input prompt text, wherein the specific part of speech includes at least adjectives and adverbs.

[0042] To model the energy consumption of large-scale inference tasks, feature extraction is performed first. In this stage, features highly correlated with inference energy consumption are selected as model inputs through analysis of the raw data. Based on theoretical knowledge, features affecting inference energy consumption are mainly divided into three categories: cue word features, model features, and hardware configuration. This invention measures the complexity of input cue words by statistically analyzing the number of adjectives, adverbs, and other lexical categories, as well as the length of the cue words. This metric can reflect the potential computational complexity of the model when parsing cue words with low computational overhead. Model features include model type and model parameters. Here, model parameters refer to the sum of all trainable weights and bias terms in the model, reflecting the model's structural scale and storage complexity. The total number is calculated by counting the number of elements in the weight matrix and bias vector of each layer. Hardware configuration refers to the platform on which the model is deployed, including different GPU models.

[0043] Although a large number of features exist, only a few are significantly correlated with inference energy consumption. Using all features as input to model inference energy consumption could lead to overfitting and slower training. Therefore, feature extraction techniques are employed when constructing input features, retaining only those strongly correlated with energy consumption. In addition to features directly obtainable from the dataset, parameter counts are derived based on the model layer number l, hidden size h, and vocabulary size V. Its calculation formula is: (The formula and physical meaning of the enhanced features are given, along with the reasons for choosing them as features.) P represents the number of trainable weights in the model, reflecting the model's representational power, storage requirements, and computational complexity. A larger P indicates that the model requires more GPU memory and incurs higher inference computation and energy consumption. This is based on the input prompt word length D and the parameter count. Extending this to include the FLOPs of the reasoning stage, the formula for their calculation is as follows: , which refers to the total number of floating-point operations required for a model to complete one forward propagation. It is used to measure the computational complexity of a model when performing inference tasks and is often used to compare the efficiency or computational cost of different models.

[0044] After adding two new features, feature selection was performed based on the Spearman correlation coefficient, with a threshold of |ρ| > 0.30 to retain variables with a moderate to high monotonic relationship with inference energy consumption. The results showed that several features were significantly correlated with inference energy consumption, including the number of response tokens, input prompt complexity, model parameters, and FLOPs during the inference phase. Furthermore, based on theoretical knowledge, hardware configuration was also included as a feature in the analysis, as it is believed to also affect energy consumption. Among these features, the number of response tokens showed the strongest correlation with inference energy consumption, significantly impacting modeling accuracy. However, in actual inference energy consumption modeling, the number of response tokens depends on the dynamic generation mechanism during the inference process; the model must process input prompts and iteratively generate tokens until the answer is completed, thus making it impossible to predict in advance. Based on this, a two-stage enhanced energy analysis model is proposed, which will be detailed in steps S200-S300.

[0045] S200: Input the multidimensional heterogeneous feature set into the first prediction model. The first prediction model is a classifier trained based on historical task data, used to map the reasoning task to a predefined interval category of response token count. The interval category is a discrete category divided by unsupervised clustering analysis of the actual response token count of historical tasks.

[0046] In some embodiments, the predefined range of response token number categories is determined using the K-means clustering algorithm, and the optimal number of categories is determined by evaluating the silhouette coefficient method.

[0047] In some embodiments, the first prediction model is a multilayer perceptron neural network classifier, whose output layer uses the Softmax activation function, corresponding to each category of the interval category.

[0048] Before predicting the number of response tokens, we first used Spearman correlation coefficients to analyze the correlation between each feature and the number of response tokens. The results show that the features most strongly correlated with the number of response tokens originate from the characteristics of the input prompts, mainly including the complexity of the prompt words and the sentence length. Furthermore, in practical large-scale model inference tasks, different models often produce different outputs for the same input prompt words. Even the same model may produce different results when deployed on different hardware platforms. Therefore, we also consider model name, model parameter size, and hardware configuration as input features.

[0049] In predicting the number of response tokens, various methods were analyzed and compared. Considering the heterogeneity of features and the uncertainty of the number of response tokens, directly using regression methods to predict their relationship is difficult. To enhance model stability and improve prediction accuracy, the regression task was restructured as a classification problem.

[0050] First, the number of response tokens is divided into several intervals by minimizing the intra-cluster variance using the K-means algorithm. The goal of K-Means is to find K cluster centers and assign all samples to the nearest cluster center, minimizing the sum of squared distances between samples within the cluster. Its mathematical objective function is given by the formula:

[0051]

[0052] in, The preset number of clusters, For the first Clusters, For the first The centroid (mean vector) of each cluster. These are sample points.

[0053] Initializing cluster centers: Since there are K cluster centers, the core step in initialization is choosing the value of K. Common methods include the elbow method and the silhouette coefficient method. The elbow method involves plotting the objective function J(SSE, sum of squared errors within the cluster) for different K values ​​and selecting the K value at the inflection point. The silhouette coefficient method requires calculating the silhouette coefficient of sample i, using the following formula:

[0054]

[0055] in: For the profile coefficient, For the sample The average distance to other samples in the same cluster. For the sample The average distance to the nearest other cluster sample.

[0056] Its value ranges from [-1, 1], with larger values ​​indicating better clustering results. The silhouette coefficient is used to calculate the inference energy consumption of the dataset, such as... Figure 2 As shown, the clustering effect is best when K=3. After determining the value of K, K samples are randomly selected as the initial centroids. During the clustering process, the sample points are the number of response tokens in the dataset, which is a one-dimensional feature.

[0057] Assign samples to the nearest cluster: for each sample point Calculate its distances to all centroids (usually Euclidean distances), and then... Assign it to the cluster of the nearest centroid.

[0058] Update cluster centers: Recalculate the centroid of each cluster, which is the mean vector of all samples in that cluster, using the following formula:

[0059]

[0060] Iterative optimization: Repeatedly assign samples to the nearest cluster and update the cluster centers until the termination conditions are met: the centroids no longer change, the maximum number of iterations is reached, and the change in the objective function J is less than a threshold. Through iterative optimization, the final number of response tokens is divided into three clusters, as follows: Figure 3 As shown: [1,262], [263,538], [539,836]. Response tokens from 1 to 262 are grouped into cluster 0, response tokens from 263 to 538 into cluster 1, and response tokens from 538 to 836 into cluster 2.

[0061] Finally, a neural network predicts a range of response token numbers based on the features of the input prompts, model parameters, and hardware configuration. This predicted range is used as an enhanced feature in the subsequent energy consumption modeling stage. The network parameters are shown in Table 1 below, and the training dataset is from the publicly available dataset ejhusom / llm-inference-energy-consumption.

[0062] Table 1

[0063]

[0064] S300: Encode the obtained interval categories to generate interval category feature vectors; fuse the interval category feature vectors with the multidimensional heterogeneous feature set to form an enhanced feature set; input the enhanced feature set into the second prediction model and output the energy consumption prediction value of the inference task; the second prediction model is a regression model trained based on the enhanced historical task data, which uses a gradient boosting decision tree as its core to model the complex nonlinear relationship between the enhanced features and energy consumption.

[0065] In some embodiments, the interval category feature vector is generated using one-hot encoding.

[0066] In some embodiments, the second prediction model is an XGBoost regression model, whose objective function is a squared loss function including a regularization term, which constrains the number and weight of the leaf nodes of the decision tree.

[0067] In some embodiments, a model training step is also included:

[0068] S0a: Collect historical reasoning task datasets, use the K-means algorithm to cluster the number of their true response tokens, forming K intervals as interval labels;

[0069] S0b: Using the multidimensional heterogeneous feature set of the historical task as input and the interval label obtained in step S0a as the target, train a neural network classifier as the first prediction model.

[0070] S0c: Use the trained first prediction model to perform forward propagation on historical task data to obtain its predicted interval category, encode the interval category and fuse it with the original feature set, and form a sample pair with the real energy consumption value;

[0071] S0d: Using the enhanced training set constructed in step S0c, train an XGBoost regression model to obtain the second prediction model.

[0072] Based on the above research findings, the predicted response token number range (0, 1, 2) is one-hot encoded and incorporated into the model as an enhancement feature. This feature is then combined with other original features for energy consumption modeling of the inference task. Using the XGBoost algorithm, a nonlinear relationship model between input features and inference stage energy consumption is established to characterize the impact of features on energy consumption. The model input includes the predicted response token number, hardware configuration, model parameters, model name, cue word length, and enhancement features such as parameter counts and inference stage FLOPs. These input variables exhibit a nonlinear correlation, and the output is the energy consumption of a large-scale language model inference task.

[0073] like Figure 4 As shown, XGBoost is a gradient boosting-based regression tree ensemble method. It employs an additive model to progressively stack base learners, growing a new regression tree in each iteration to fit the residuals left from the previous round's predictions, thereby continuously reducing the overall error. To prevent overfitting in multiple iterations, XGBoost explicitly incorporates structured regularization into the objective, imposing constraints on the number of leaves and the output size of leaf nodes. Combined with mechanisms such as learning rate, column sampling, minimum leaf sample count, maximum depth, and early stopping, it balances expressive power and generalization performance. Based on these characteristics, XGBoost can automatically capture complex nonlinear and feature interactions. When used for LLM inference energy consumption modeling, it can robustly characterize the combined impact of multiple features on energy consumption, including the number of response tokens, hardware configuration, model parameters, model name, cue word length, parameter count, and FLOPs during the inference stage. The following is a detailed introduction:

[0074] The training data is as follows:

[0075]

[0076] in, For the sample size, For the first Features related to inference energy consumption in each sample This represents the corresponding inference energy consumption.

[0077] XGBoost uses an additive combination of multiple regression trees as a predictor:

[0078]

[0079] in, For the sample The predicted energy consumption value; This represents the number of regression trees; Indicates the first A regression tree in the input Output at; Let be the function space of the regression tree.

[0080] Objective function of the inference energy consumption model:

[0081]

[0082] in, For point loss, the squared error can be taken in regression scenarios. That is, the difference between the predicted inference energy consumption and the actual value; is the regularization term of the k-th tree; T is the number of leaf nodes of that tree; The output weight of the j-th leaf node; To punish the number of leaves and encourage simple structures; The L2 regularization coefficient is used to improve stability and suppress overfitting.

[0083] To evaluate the prediction accuracy of the proposed two-stage enhanced energy analysis model compared to other models, we used four evaluation metrics on the test set: mean absolute error (MAE), mean squared error (MSE), root mean square error (RMSE), and coefficient of determination (R²). We set up the following three comparison scenarios:

[0084]

[0085]

[0086]

[0087] Example 1: Using only fully connected neural networks to model inference energy consumption.

[0088] Example 2: A two-stage model is used to model inference energy consumption. First, a classification neural network is used to predict the number of response tokens, and then a Transformer is used to model inference energy consumption.

[0089] Example 3: A two-stage model is used to model inference energy consumption. First, a classification neural network is used to predict the number of response tokens, and then a fully connected neural network is used to model inference energy consumption.

[0090] The energy consumption model proposed in this solution, compared with other models, yields accuracy results such as... Figure 5 As shown:

[0091] Where (a), (b), (c), and (d) correspond to Example 1, Example 2, Example 3, and the method proposed in this invention, respectively;

[0092] Table 2 shows a comparison of the error evaluation metrics of the model proposed in this solution with other models:

[0093] Table 2

[0094]

[0095] Figure 5 The prediction results of the proposed method and three comparative schemes are presented. Compared with other methods, the predicted values ​​proposed in this paper are closer to the true values ​​and have lower dispersion, indicating a higher degree of agreement with actual observation data. This improvement is mainly due to the two-stage modeling strategy and XGBoost's powerful ability to capture nonlinear feature correlations and complex interactions, effectively reducing the uncertainty of inferred energy consumption estimation. As shown in Table 1, the proposed model outperforms the single-stage model on all evaluation metrics. Compared with Example 1, the proposed method improves the MAE by 89% and the R2 by 79%. Compared with Examples 2 and 3, the MAE is reduced by 39% and 67%, respectively, while the R2 is improved by 16% and 15%, respectively. These results fully verify the effectiveness and superiority of the proposed method.

[0096] Secondly, this application proposes a two-stage large-scale language model energy consumption analysis system for reasoning tasks, including: a feature engineering module, a first prediction module, a second prediction module, and a data bus and processing logic module;

[0097] The feature engineering module is used to acquire initial known information about the inference task to be predicted, and to extract and construct a multidimensional heterogeneous feature set X. The multidimensional heterogeneous feature set includes: cue word length D, cue word complexity C, total model parameter count P, theoretical number of floating-point operations per forward propagation IC, and hardware configuration identifier H. The total model parameter count P is calculated based on the model architecture parameters. Where l is the number of model layers, h is the hidden size, and V is the vocabulary size, the theoretical number of floating-point operations IC is calculated according to the formula. Calculated;

[0098] The first prediction module is used to input the multidimensional heterogeneous feature set into the first prediction model. The first prediction model is a classifier trained based on historical task data. It is used to map the reasoning task to a predefined interval category of response token count. The interval category is a discrete category divided by unsupervised clustering analysis of the actual response token count of historical tasks.

[0099] The second prediction module is used to encode the obtained interval categories to generate interval category feature vectors; to fuse the interval category feature vectors with the multidimensional heterogeneous feature set to form an enhanced feature set; to input the enhanced feature set into the second prediction model and output the energy consumption prediction value of the inference task; the second prediction model is a regression model trained based on the enhanced historical task data, which uses a gradient boosting decision tree as its core to model the complex nonlinear relationship between the enhanced features and energy consumption;

[0100] The data bus and processing logic module are used to transmit the output of the feature engineering module to the first prediction module, encode the output of the first prediction module and fuse it with the original features, and then transmit it to the second prediction module.

[0101] Thirdly, this application proposes an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0102] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described above.

[0103] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0104] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0105] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0106] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0107] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0108] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0109] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0110] The above are merely preferred embodiments of the present invention. It should be noted that any modifications and improvements made by those skilled in the art without departing from the present technical solution should also be considered to fall within the scope of protection claimed by the present solution.

Claims

1. A two-stage energy consumption analysis method for large-scale language models oriented towards reasoning tasks, characterized in that, Includes the following steps: Obtain the initial known information of the inference task to be predicted, and extract and construct a multidimensional heterogeneous feature set; The multidimensional heterogeneous feature set includes: prompt word length D, prompt word complexity C, total model parameter count P, theoretical number of floating-point operations per forward propagation IC, and hardware configuration identifier H; wherein, the total model parameter count P is calculated based on the model architecture parameters. , l For the number of model layers, h To hide size, V Given the vocabulary size, the theoretical number of floating-point operations (IC) is calculated according to the formula. The complexity C of the prompt words is calculated by statistically analyzing the proportion of words with specific parts of speech in the input prompt text, wherein the specific parts of speech include at least adjectives and adverbs. The multidimensional heterogeneous feature set is input into the first prediction model, which is a classifier trained based on historical task data. The classifier is used to map the reasoning task to a predefined interval category of response token count. The interval category is a discrete category divided by unsupervised clustering analysis of the actual response token count of historical tasks. The obtained interval categories are encoded to generate interval category feature vectors; the interval category feature vectors are fused with the multidimensional heterogeneous feature set to form an enhanced feature set; the enhanced feature set is input into the second prediction model to output the energy consumption prediction value of the inference task; the second prediction model is a regression model trained based on the enhanced historical task data, which uses a gradient boosting decision tree as its core to model the complex nonlinear relationship between the enhanced features and energy consumption. The second prediction model is an XGBoost regression model, whose objective function is a squared loss function containing a regularization term. The regularization term constrains the number of leaf nodes and weights of the decision tree. The model training steps are also included. S0a: Collect historical reasoning task datasets, use the K-means algorithm to cluster the number of their true response tokens, forming K intervals as interval labels; S0b: Using the multidimensional heterogeneous feature set of the historical task as input and the interval label obtained in step S0a as the target, train a neural network classifier as the first prediction model. S0c: Use the trained first prediction model to perform forward propagation on historical task data to obtain its predicted interval category, encode the interval category and fuse it with the original feature set, and form a sample pair with the real energy consumption value; S0d: Using the enhanced training set constructed in step S0c, train an XGBoost regression model to obtain the second prediction model.

2. The method according to claim 1, characterized in that: The predefined range of response token numbers is determined using the K-means clustering algorithm, and the optimal number of categories is determined by evaluating the silhouette coefficient method.

3. The method according to claim 2, characterized in that: The first prediction model is a multilayer perceptron neural network classifier, whose output layer uses the Softmax activation function, corresponding to each category of the interval category.

4. The method according to claim 3, characterized in that: The interval category feature vectors are generated using one-hot encoding.

5. A two-stage energy consumption analysis system for large-scale language models oriented towards reasoning tasks, the system performing steps for implementing the method as described in any one of claims 1-4, characterized in that, include: The module includes a feature engineering module, a first prediction module, a second prediction module, and a data bus and processing logic module. The feature engineering module is used to obtain the initial known information of the inference task to be predicted, and to extract and construct a multidimensional heterogeneous feature set X. The multidimensional heterogeneous feature set includes: prompt word length D, prompt word complexity C, total model parameter count P, theoretical number of floating-point operations per forward propagation IC, and hardware configuration identifier H; wherein, the total model parameter count P is calculated based on the model architecture parameters. , l For the number of model layers, h To hide size, V Given the vocabulary size, the theoretical number of floating-point operations (IC) is calculated according to the formula. The complexity C of the prompt words is calculated by statistically analyzing the proportion of words with specific parts of speech in the input prompt text, wherein the specific parts of speech include at least adjectives and adverbs. The first prediction module is used to input the multidimensional heterogeneous feature set into the first prediction model. The first prediction model is a classifier trained based on historical task data. It is used to map the reasoning task to a predefined interval category of response token count. The interval category is a discrete category divided by unsupervised clustering analysis of the actual response token count of historical tasks. The second prediction module is used to encode the obtained interval categories to generate interval category feature vectors; fuse the interval category feature vectors with the multidimensional heterogeneous feature set to form an enhanced feature set; input the enhanced feature set into the second prediction model to output the energy consumption prediction value of the inference task; the second prediction model is a regression model trained based on the enhanced historical task data, which uses a gradient boosting decision tree as its core to model the complex nonlinear relationship between the enhanced features and energy consumption. The second prediction model is an XGBoost regression model, whose objective function is a squared loss function containing a regularization term, which constrains the number of leaf nodes and weights of the decision tree. The module also includes model training steps. S0a: Collect historical reasoning task datasets, use the K-means algorithm to cluster the number of their true response tokens, forming K intervals as interval labels; S0b: Using the multidimensional heterogeneous feature set of the historical task as input and the interval label obtained in step S0a as the target, train a neural network classifier as the first prediction model. S0c: Use the trained first prediction model to perform forward propagation on historical task data to obtain its predicted interval category, encode the interval category and fuse it with the original feature set, and form a sample pair with the real energy consumption value; S0d: Using the enhanced training set constructed in step S0c, train an XGBoost regression model to obtain the second prediction model; The data bus and processing logic module are used to transmit the output of the feature engineering module to the first prediction module, encode the output of the first prediction module and fuse it with the original features, and then transmit it to the second prediction module.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method as described in any one of claims 1-4.

Images