Transform-based server energy consumption prediction method
Through the combination of PCA principal component analysis and Transformer model, the server resource utilization characteristics are extracted, and the problem of large amount of model training in the existing technology is solved, efficient server energy consumption prediction is achieved, and operation and maintenance costs are reduced.
Patent Information
- Application Number
- CN202510338186.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-21
- Publication Date
- 2025-07-18
AI Technical Summary
The existing server energy consumption prediction technology needs to train models separately for different load states, increasing the model training volume and enterprise operation and maintenance costs.
The PCA principal component analysis method is used to extract server resource utilization characteristics, and the Transformer model is used to predict energy consumption. By training and fine-tuning the data set, the model is optimized, and the training volume is reduced and the prediction accuracy is improved.
Accurate energy consumption prediction under zero load, low load and high load states are achieved, which reduces model training volume, reduces enterprise operation and maintenance costs, and provides strong energy consumption prediction results support.
Smart Images
Figure CN120336138A_ABST
Abstract
Description
Technical Field
[0001] The present invention discloses a server energy consumption prediction method based on Transformer, which relates to the fields of energy consumption prediction and artificial intelligence. Background Art
[0002] With the rapid progress of information technology and the wide application of technologies such as cloud computing, big data, and artificial intelligence, data centers, as the core infrastructure for data storage and computing, are expanding in number and scale. However, this has also led to the increasingly prominent problem of energy consumption in data centers. As an important part of data centers, servers have a relatively high proportion of energy consumption. Nowadays, a large number of researchers and enterprises have begun to pay attention to and invest resources in the research and development of server energy consumption prediction technologies. Most of these technologies are based on advanced algorithms such as machine learning and deep learning. By collecting the system resource utilization rate and real-time energy consumption data of servers in different load states, a prediction model is constructed, and the prediction accuracy and practicability are improved through segmented training of the model. However, the method of dividing into levels requires separate training of models for different load states, which greatly increases the training volume of the model and increases the enterprise operation and maintenance cost. Summary of the Invention
[0003] Aiming at the problems of the prior art, the present invention provides a server energy consumption prediction method based on Transformer, which is used to improve the prediction accuracy of server energy consumption, facilitate operation and maintenance personnel to optimize the design of servers with high energy consumption in a timely manner, and reduce the enterprise operation and maintenance cost.
[0004] The specific solution proposed by the present invention is as follows:
[0005] The present invention provides a server energy consumption prediction method based on Transformer, including:
[0006] Step 1: Collect the energy consumption data of the server and the corresponding server resource utilization rate data respectively under zero load, low load, and high load states.
[0007] Step 2: Perform data preprocessing: Clean the collected server resource utilization rate data and energy consumption data, and perform normalization processing.
[0008] Step 3: Use the PCA principal component analysis method to extract features from the preprocessed server resource utilization rate.
[0009] Step 4: Create a Transformer model. Among the extracted feature data, select the energy consumption-related data under the low load state with a larger data volume as the training dataset of the Transformer model, and select the energy consumption-related datasets under the zero load state and the high load state as the fine-tuning datasets of the Transformer model. Use the test set to perform performance testing on the Transformer energy consumption model after training and fine-tuning.
[0010] Step 5: Train the Transformer model using the training dataset so that the model can predict the server energy consumption under the low load state. Fine-tune the Transformer model using the fine-tuning dataset so that the model has the ability to predict the server energy consumption under the zero load and high load states. Use the test set to test the prediction performance of the Transformer model.
[0011] Step 6: Input the server resource utilization data to be predicted for energy consumption into the Transformer model for server energy consumption prediction. If the predicted energy consumption situation is too high, take measures to reduce the load usage. If the predicted energy consumption situation is normal, maintain the existing load operation state.
[0012] Furthermore, the conditions for determining the zero load, low load, and high load states in step 1 of the server energy consumption prediction method based on Transformer include:
[0013] Take the working state when the total CPU utilization rate of the server is less than 3% as the zero load state.
[0014] Take the working state when the total CPU utilization rate of the server is greater than or equal to 3% and less than 50% as the low load state.
[0015] Take the working state when the total CPU utilization rate of the server is greater than or equal to 50% and less than or equal to 100% as the high load state.
[0016] The resource utilization data includes CPU utilization rate, memory utilization rate, network bandwidth utilization rate, disk utilization rate, system average load, and system response time.
[0017] Furthermore, in step 2 of the server energy consumption prediction method based on Transformer, use the following formula:
[0018]
[0019] Perform normalization processing, where x represents the original data, x' represents the normalized data, x max represents the maximum value of this piece of data, and x min represents the minimum value of this piece of data.
[0020] Further, in step 3 of the Transformer-based server energy consumption prediction method, the PCA principal component analysis method is used to extract features from the preprocessed server resource utilization rate, including:
[0021] Let the preprocessed server resource utilization rate data be X and the energy consumption data be Y, then [X, Y] = [(X 零负载 , X 低负载 , X 高负载 ), (X 零负载 , Y 低负载 , Y 高负载 ). Perform principal component feature extraction on the resource utilization rate data X. First, calculate the covariance matrix of the sample XX T , and perform eigenvalue decomposition on the covariance matrix to solve for the corresponding n eigenvalues (λ1, λ2,..., λ n ) and eigenvectors (w1, w2,..., w n ), where λ1 ≥ λ2 ≥... ≥ λ n ; then, according to the cumulative contribution rate of the eigenvalues, select the eigenvectors (w1, w2,..., w n′ ) corresponding to the first n' eigenvalues to form the eigenvector matrix W, and use W to perform principal component feature extraction on the resource utilization rate data X. The cumulative contribution rate of the eigenvalues is shown in Equation (2):
[0022]
[0023] where t is the principal component proportion threshold,
[0024] The new sample data after feature extraction is Z = W T X.
[0025] Further, in step 4 of the Transformer-based server energy consumption prediction method, a Transformer model is created. The multi-head attention mechanism is adopted to enhance the feature representation ability of the model. Multiple independent attention heads are introduced, so that the Transformer model can learn richer feature representations from different subspaces. The calculation formula is as follows:
[0026] MultiHead(Q, K, V) = Concat(head1, head2,..., head h )W O Equation (4)
[0027] where Q, K, and V represent the query, key, and value matrices respectively, i = 1, 2,..., h, h is the number of attention heads, are learnable parameters.
[0028] Further, in step 5 of the Transformer-based server energy consumption prediction method, the mean squared error RMSE is used as an evaluation index for the model performance, and the formula is as follows:
[0029]
[0030] where N represents the number of samples in the test set, y i represents the true value of the test output, represents the predicted value.
[0031] The present invention also provides a Transformer-based server energy consumption prediction device, including an acquisition module, a preprocessing module, a feature extraction module, a model data set management module, a training module, and a prediction module.
[0032] The acquisition module collects the energy consumption data of the server and the corresponding server resource utilization data respectively under zero load, low load, and high load states.
[0033] The preprocessing module performs data preprocessing: cleaning the collected server resource utilization data and energy consumption data, and performing normalization processing.
[0034] The feature extraction module uses the PCA principal component analysis method to extract features from the preprocessed server resource utilization.
[0035] The model data set management module creates a Transformer model. Among the extracted feature data, the energy consumption-related data under the low load state with a larger data volume is selected as the training data set of the Transformer model, and the energy consumption-related data sets under the zero load state and the high load state are selected as the fine-tuning data sets of the Transformer model. The performance of the trained and fine-tuned Transformer energy consumption model is tested using the test set.
[0036] The training module trains the Transformer model using the training data set so that the model can predict the server energy consumption under the low load state, fine-tunes the Transformer model using the fine-tuning data set so that the model has the ability to predict the server energy consumption under the zero load and high load states, and tests the prediction performance of the Transformer model using the test set.
[0037] The prediction module inputs the server resource utilization data to be predicted for energy consumption into the Transformer model for server energy consumption prediction. If the predicted energy consumption situation is too high, measures to reduce the load usage are taken. If the predicted energy consumption situation is normal, the existing load operation state is maintained.
[0038] The present invention also provides a computer-readable medium, characterized in that computer instructions are stored on the computer-readable medium, and when the computer instructions are executed by a processor, the processor executes the above-mentioned Transformer-based server energy consumption prediction method.
[0039] The advantages of the present invention are as follows:
[0040] By using the principal component analysis method to extract key feature information from the resource utilization data with unknown correlation relationships among multiple variables, the redundancy of the data is reduced, the computational amount of the model is lowered, the state data with the largest amount of data among the three states of zero load, low load, and high load is used for the training of the Transformer energy consumption model, and then the data in the remaining two states is used to fine-tune the model, so as to realize the prediction of the server energy consumption in the zero load, low load, and high load states by the prediction model, greatly reducing the training amount of the model. The proposed energy consumption prediction model can accurately predict the energy consumption of the server, providing strong auxiliary support for the energy consumption prediction results for the operation and maintenance personnel, facilitating the timely formulation of optimization plans, and reducing the enterprise operation and maintenance costs. Description of the Drawings
[0041] Figure 1 It is a schematic flowchart of the method of the present invention. Detailed Embodiments
[0042] The present invention will be further described below in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the present invention and be able to implement it, but the embodiments given are not intended to limit the present invention.
[0043] Embodiment 1
[0044] The present invention provides a Transformer-based server energy consumption prediction method, including:
[0045] Step 1: Collect the energy consumption data of the server and the corresponding server resource utilization data respectively in the zero load, low load, and high load states.
[0046] The resource utilization data includes but is not limited to CPU utilization rate, memory utilization rate, network bandwidth utilization rate, disk utilization rate, system average load, system response time, etc.
[0047] The conditions for judging the zero load, low load, and high load states are as follows: The working state when the total CPU utilization rate of the server is less than 3% is taken as the zero load state, the working state when the total CPU utilization rate of the server is greater than or equal to 3% and less than 50% is taken as the low load state, and the working state when the total CPU utilization rate of the server is greater than or equal to 50% and less than or equal to 100% is taken as the high load state.
[0048] Step 2: Perform data preprocessing: Clean the collected server resource utilization data and energy consumption data, and perform normalization processing.
[0049] The normalization process is as follows:
[0050]
[0051] Among them, x represents the original data, x' represents the normalized data, x max represents the maximum value of this piece of data, x min represents the minimum value of this piece of data.
[0052] Step 3: Use the PCA principal component analysis method to extract features from the preprocessed server resource utilization.
[0053] The feature extraction process is as follows: Assume that the preprocessed server resource utilization data X and energy consumption data Y are [X, Y] = [(X 零负载 , X 低负载 , X 高负载 ), (X 零负载 , Y 低负载 , Y 高负载 ). Perform principal component feature extraction on the resource utilization data X. First, calculate the covariance matrix of the sample XX T , and perform eigenvalue decomposition on the covariance matrix to solve for the corresponding n eigenvalues (λ1, λ2,..., λ n ) and eigenvectors (w1, w2,..., w n ), where λ1 ≥ λ2 ≥... ≥ λ n ; Secondly, according to the cumulative contribution rate of the eigenvalues, select the eigenvectors (w1, w2,..., w n′ ) corresponding to the first n' eigenvalues to form the eigenvector matrix W, which is used to perform principal component feature extraction on the resource utilization data X. The cumulative contribution rate of the eigenvalues is shown in Equation (2); finally, the new sample data after feature extraction is Z = W T X.
[0054] The cumulative contribution rate of the eigenvalues is as follows:
[0055]
[0056] Among them, t is the principal component proportion threshold.
[0057] Step 4: Create a Transformer model. Among the extracted feature data, select the energy consumption-related data in the low-load state with a larger amount of data as the training dataset of the Transformer model, and select the energy consumption-related datasets in the zero-load state and high-load state as the fine-tuning datasets of the Transformer model. Use the test set to perform a performance test on the Transformer energy consumption model after training and fine-tuning.
[0058] Among them, the energy consumption-related data after feature extraction can be expressed as [Z, Y] = [(Z 零负载 , Z 低负载 , Z 高负载 ), (Y 零负载 , Y 低负载 , Y 高负载 )].
[0059] Create a Transformer model. The formula for the self-attention mechanism is as follows:
[0060]
[0061] Among them, Q, K, and V represent the query, key, and value matrices respectively, and dk is the dimension of the key matrix.
[0062] Adopt the multi-head attention mechanism to enhance the feature representation ability of the model. Introduce multiple independent attention heads, so that the Transformer model can learn more rich feature representations from different subspaces. The calculation formula is as follows:
[0063] MultiHead(Q, K, V) = Concat(head1, head2,..., head h )W O Equation (4)
[0064] Among them, Q, K, and V represent the query, key, and value matrices respectively, i = 1, 2,..., h, where h is the number of attention heads, is a learnable parameter.
[0065] Step 5: Use the training dataset to train the Transformer model so that the model can predict the server energy consumption in the low-load state. Use the fine-tuning dataset to fine-tune the Transformer model so that the model has the ability to predict the server energy consumption in the zero-load and high-load states. Use the test set to test the prediction performance of the Transformer model, and use the root mean square error RMSE as the evaluation index of the model performance. The formula is as follows:
[0066]
[0067] where N represents the number of samples in the test set, and y i represents the true value of the test output, represents the predicted value.
[0068] Step 6: Input the server resource utilization data to be predicted for energy consumption into the Transformer model for server energy consumption prediction. If the predicted energy consumption situation is too high, measures to reduce the load usage are taken. If the predicted energy consumption situation is normal, the existing load operation state is maintained.
[0069] Embodiment 2
[0070] The present invention also provides a server energy consumption prediction device based on Transformer, including a collection module, a preprocessing module, a feature extraction module, a model dataset management module, a training module, and a prediction module.
[0071] The collection module collects the energy consumption data of the server and the corresponding server resource utilization data respectively under the zero-load, low-load, and high-load states.
[0072] The preprocessing module performs data preprocessing: cleaning the collected server resource utilization data and energy consumption data, and performing normalization processing.
[0073] The feature extraction module uses the PCA principal component analysis method to extract features from the preprocessed server resource utilization.
[0074] The model dataset management module creates a Transformer model. Among the extracted feature data, the energy consumption-related data in the low-load state with a larger data volume is selected as the training dataset of the Transformer model, and the energy consumption-related datasets in the zero-load state and the high-load state are selected as the fine-tuning datasets of the Transformer model. The performance of the trained and fine-tuned Transformer energy consumption model is tested using the test set.
[0075] The training module trains the Transformer model using the training dataset so that the model can predict the server energy consumption situation in the low-load state, fine-tunes the Transformer model using the fine-tuning dataset so that the model has the ability to predict the server energy consumption in the zero-load and high-load states, and tests the prediction performance of the Transformer model using the test set.
[0076] The prediction module inputs the server resource utilization data to be predicted for energy consumption into the Transformer model for server energy consumption prediction. If the predicted energy consumption situation is too high, measures to reduce the load usage are taken. If the predicted energy consumption situation is normal, the existing load running state is maintained.
[0077] Regarding the information interaction and execution process among the modules in the above device, since they are based on the same concept as the method embodiment of the present invention, the specific content can be referred to the description in the method embodiment of the present invention and will not be elaborated here.
[0078] Similarly, the device of the present invention extracts key feature information from the resource utilization data with unknown correlation relationships among multiple variables by using the principal component analysis method, reduces data redundancy, and reduces the computational amount of the model. The state data with the largest amount of data among the three states of zero load, low load, and high load is used for the training of the Transformer energy consumption model, and then the data in the remaining two states is used to fine-tune the model, so as to realize the prediction of the server energy consumption in the zero load, low load, and high load states by the prediction model, greatly reducing the training amount of the model. The proposed energy consumption prediction model can accurately predict the energy consumption situation of the server, provide strong auxiliary support for the energy consumption prediction results for operation and maintenance personnel, facilitate the timely formulation of optimization plans, and reduce the enterprise operation and maintenance costs.
[0079] It should be noted that not all steps and modules in the above processes and device structures are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted according to needs. The system structures described in the above embodiments can be physical structures or logical structures. That is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities respectively, or some components in multiple independent devices can be jointly implemented.
[0080] Embodiment 3
[0081] The present invention also provides a computer-readable medium, characterized in that computer instructions are stored on the computer-readable medium, and when the computer instructions are executed by a processor, the processor executes the server energy consumption prediction method based on Transformer. Specifically, a system or device equipped with a storage medium can be provided, and software program codes for implementing the functions of any one of the above embodiments are stored on the storage medium, and the computer (or CPU or MPU) of the system or device reads and executes the program codes stored in the storage medium.
[0082] In this case, the program code read from the storage medium itself can implement the functions of any one of the above embodiments, so the program code and the storage medium storing the program code constitute a part of the present invention.
[0083] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Optionally, the program code can be downloaded from a server computer via a communication network.
[0084] In addition, it should be clear that not only can the functions of any one of the above embodiments be realized by executing the program code read by the computer, but also by causing an operating system or the like operating on the computer based on the instructions of the program code to complete part or all of the actual operations.
[0085] In addition, it can be understood that the program code read from the storage medium is written into the memory provided in the expansion board inserted into the computer or into the memory provided in the expansion unit connected to the computer, and then based on the instructions of the program code, the CPU or the like installed on the expansion board or the expansion unit is caused to execute part and all of the actual operations, thereby realizing the functions of any one of the above embodiments.
[0086] The above-described embodiments are merely preferred embodiments given to fully illustrate the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or transformations made by those skilled in the art on the basis of the present invention are all within the protection scope of the present invention. The protection scope of the present invention shall be subject to the claims.
Claims
1. A server energy consumption prediction method based on Transformer, characterized in that Including: Step 1: Collect the energy consumption data of the server and the corresponding server resource utilization data under zero load, low load, and high load states respectively. Step 2: Perform data preprocessing: Clean the collected server resource utilization data and energy consumption data, and perform normalization processing. Step 3: Use the PCA principal component analysis method to extract features from the preprocessed server resource utilization. Step 4: Create a Transformer model. Among the extracted feature data, select the energy consumption-related data under the low load state with a larger data volume as the training data set of the Transformer model, select the energy consumption-related data sets under the zero load state and the high load state as the fine-tuning data sets of the Transformer model, and use the test set to perform performance testing on the Transformer energy consumption model after training and fine-tuning. Step 5: Use the training data set to train the Transformer model so that the model can predict the server energy consumption under the low load state, use the fine-tuning data set to fine-tune the Transformer model so that the model has the ability to predict the server energy consumption under the zero load and high load states, and use the test set to test the prediction performance of the Transformer model. Step 6: Input the server resource utilization data to be predicted for energy consumption into the Transformer model for server energy consumption prediction. If the predicted energy consumption situation is too high, take measures to reduce the load usage. If the predicted energy consumption situation is normal, maintain the existing load running state.
2. The method for predicting server energy consumption based on Transformer according to claim 1, wherein The conditions for judging zero load, low load, and high load states in Step 1 include: Take the working state when the total CPU utilization rate of the server is less than 3% as the zero load state. Take the working state when the total CPU utilization rate of the server is greater than or equal to 3% and less than 50% as the low load state. Take the working state when the total CPU utilization rate of the server is greater than or equal to 50% and less than or equal to 100% as the high load state. The resource utilization data includes CPU utilization rate, memory utilization rate, network bandwidth utilization rate, disk utilization rate, system average load, and system response time.
3. The method for predicting server energy consumption based on Transformer according to claim 1, characterized in that In Step 2, use the following formula: Perform normalization, where x represents the original data, x' represents the normalized data, x max represents the maximum value of this data, x min represents the minimum value of this data.
4. The method for predicting the energy consumption of a server based on Transformer according to claim 1, wherein In Step 3, use the PCA principal component analysis method to extract features from the preprocessed server resource utilization, including: Let the preprocessed server resource utilization data be X and the energy consumption data be Y. Then, [X, Y] = [(X 零负载 , X 低负载 , X 高负载 ), (Y 零负载 , Y 低负载 , Y 高负载 )]. Perform principal component feature extraction on the resource utilization data X. First, calculate the covariance matrix of the sample XX T , and perform eigenvalue decomposition on the covariance matrix to solve for the corresponding n eigenvalues (λ1, λ2,..., λ n ) and eigenvectors (w1, w2,..., w n ), where λ1 ≥ λ2 ≥ … ≥ λ n ; then, according to the cumulative contribution rate of the eigenvalues, select the eigenvectors (w1, w2,..., w n′ ) corresponding to the first n' eigenvalues to form the eigenvector matrix W, and use W to perform principal component feature extraction on the resource utilization data X. The cumulative contribution rate of the eigenvalues is shown in Equation (2): where t is the principal component proportion threshold. The new sample data after feature extraction is Z = W T X.
5. The method for predicting server energy consumption based on Transformer according to claim 1, characterized in that In Step 4, create a Transformer model, adopt the multi-head attention mechanism to enhance the feature representation ability of the model, introduce multiple independent attention heads, so that the Transformer model can learn richer feature representations from different subspaces. The calculation formula is as follows: MultiHead(Q, K, V) = Concat(head1, head2,..., head h )W o Equation (4) where Q, K, and V represent the query, key, and value matrices respectively, h is the number of attention heads, are learnable parameters.
6. The method for predicting server energy consumption based on Transformer according to claim 1, characterized in that In Step 5, use the mean square error RMSE as the evaluation index of the model performance. The formula is as follows: where N represents the number of samples in the test set, and y i represents the true value of the test output, and represents the predicted value.
7. The Transformer-based server energy consumption prediction device is characterized in that Including a collection module, a preprocessing module, a feature extraction module, a model data set management module, a training module, and a prediction module. The collection module collects the energy consumption data of the server and the corresponding server resource utilization data under zero load, low load, and high load states respectively. The preprocessing module performs data preprocessing: cleaning the collected server resource utilization data and energy consumption data, and performing normalization processing. The feature extraction module uses the PCA principal component analysis method to extract features from the preprocessed server resource utilization. The model dataset management module creates a Transformer model. Among the extracted feature data, it selects the energy consumption-related data in the low-load state with a relatively large amount of data as the training dataset of the Transformer model, selects the energy consumption-related datasets in the zero-load state and the high-load state as the fine-tuning datasets of the Transformer model, and uses the test set to perform a performance test on the Transformer energy consumption model after training and fine-tuning. The training module uses the training dataset to train the Transformer model so that the model can predict the server energy consumption in the low-load state, uses the fine-tuning dataset to fine-tune the Transformer model so that the model has the ability to predict the server energy consumption in the zero-load and high-load states, and uses the test set to test the prediction performance of the Transformer model. The prediction module inputs the server resource utilization data to be predicted for energy consumption into the Transformer model for server energy consumption prediction. If the predicted energy consumption situation is too high, measures to reduce the load usage are taken. If the predicted energy consumption situation is normal, the existing load operation state is maintained.
8. A computer-readable medium, characterized in that Computer instructions are stored on the computer-readable medium, and when the computer instructions are executed by a processor, the processor executes the Transformer-based server energy consumption prediction method according to any one of claims 1 to 6.