Data center electric quantity prediction method based on machine learning algorithm
By applying the power prediction method based on machine learning algorithms in the data center, an efficient prediction model is built, which solves the problem that traditional power prediction methods cannot accurately capture the energy consumption change laws of data centers, and achieves higher prediction accuracy and energy consumption reduction.
Patent Information
- Application Number
- CN202411698758.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-26
- Publication Date
- 2025-04-01
AI Technical Summary
Traditional power prediction methods rely on simple statistical models and cannot accurately capture the complex energy consumption changes in data centers, resulting in low accuracy of prediction results.
Using a data center power prediction method based on machine learning algorithms, a variety of machine learning models (such as linear regression, decision trees, random forests, neural networks, etc.) are built by obtaining data from the monitoring device of the data center, preprocessing and feature engineering, and a variety of machine learning models (such as linear regression, decision trees, random forests, neural networks, etc.) are selected through cross-validation for training and evaluation.
It achieves higher accuracy and stability of power forecasting, effectively reduces the energy consumption of the data center and improves operational efficiency, which is manifested as a reduction of energy consumption by 15%, saving about 1 million yuan in electricity bills per year, and reducing PUE from 1.6 to 1.4.
Smart Images
Figure BDA0005152740190000041 
Figure BDA0005152740190000042 
Figure BDA0005152740190000045
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of fault diagnosis of electric energy conversion devices, and relates to a method for diagnosing faults of a charging pile grid-side converter, and in particular to a method for diagnosing faults of a charging pile grid-side converter based on multi-dimensional feature extraction. Background Art
[0002] With the rapid development of information technology, data centers, as core facilities for information processing, have become increasingly energy-intensive. The energy consumption of data centers not only occupies a large amount of electricity resources, but also generates a large amount of carbon emissions, which has a negative impact on the environment. According to statistics, the total energy consumption of data centers around the world has exceeded 1% of global electricity consumption, and this proportion is increasing year by year. Therefore, improving the energy efficiency of data centers and reducing operating costs have become key issues of concern to the industry.
[0003] In order to meet this challenge, many companies and research institutions have begun to explore various methods to optimize the energy consumption management of data centers. Among them, power forecasting, as an effective means, can help data center managers understand future power demand in advance, so as to take corresponding measures, such as adjusting the operating status of equipment, optimizing the cooling system, etc., to achieve the goal of energy saving and emission reduction.
[0004] However, traditional power forecasting methods often rely on simple statistical models and cannot accurately capture the complex energy consumption changes in data centers, resulting in low accuracy of forecast results.
[0005] Therefore, in order to solve the above problems, the present invention proposes a data center power prediction method based on a machine learning algorithm.
[0006] After searching, no public documents of the prior art identical or similar to the present invention were found. Summary of the invention
[0007] The purpose of the present invention is to overcome the shortcomings of the prior art and propose a data center power prediction method based on a machine learning algorithm. By constructing an efficient prediction model, a scientific basis is provided for the energy management of the data center, thereby achieving the goal of energy conservation and emission reduction and improving the operational efficiency of the data center.
[0008] The present invention solves the practical problem by adopting the following technical solutions:
[0009] A data center power prediction method based on a machine learning algorithm comprises the following steps:
[0010] Step 1: Obtain data from various monitoring devices in the data center;
[0011] Step 2: preprocess the data obtained in step 1;
[0012] Step 3: Based on the data preprocessed in step 2, perform feature engineering and extract features;
[0013] Step 4: Based on the features extracted in step 3, multiple machine learning algorithms are used to build and train a data center power prediction model, and the optimal model is selected through cross-validation;
[0014] Step 5: Evaluate the data center power prediction model constructed by the neural network method;
[0015] Step 6: Apply the trained model to actual scenarios to achieve automated prediction and obtain data center power prediction results.
[0016] Furthermore, the step 1 obtains the following types of data from various monitoring devices in the data center:
[0017] (1) Historical electricity consumption data: including total electricity consumption in each time period and electricity consumption of each device;
[0018] (2) Equipment operation status data: including operation status information of servers, storage devices, and network devices;
[0019] (3) Environmental data: including temperature, humidity, and light intensity environmental parameters inside and outside the data center;
[0020] (4) External influencing factors data: including weather forecast, holiday information, and user visit volume.
[0021] Moreover, the specific steps of step 2 include:
[0022] (1) Data cleaning: remove outliers and noise from the data and fill in missing values;
[0023] (2) Data standardization: converting data to the same scale;
[0024] (3) Data normalization: converting data into values between 0 and 1;
[0025] Moreover, the specific steps of step 3 include:
[0026] (1) Feature selection:
[0027] Using the filtering method, the historical power consumption of the data center during the flat, peak and valley periods is selected as the features;
[0028] (2) Feature structure:
[0029] The time feature method is used and the ISO 8601 format is adopted as the timestamp of the practice feature. The timestamp is divided into quarters. At the same time, the aggregation feature method is used to aggregate the quarters and the peak, valley and normal period electricity in the time feature to calculate the average, maximum and minimum values of the peak, valley and normal period electricity in different quarters.
[0030] (3) Feature conversion:
[0031] The eigenvalues are transformed into polynomials using polynomial transformation.
[0032] Moreover, the specific steps of using multiple machine learning algorithms to construct and train the data center power prediction model in step 4 include:
[0033] (1) Use linear regression method to build a data center power prediction model;
[0034] Applicable to data with strong linear relationship, the model form is:
[0035] y=β0+β1x1+β2x2+…+β n x n
[0036] Among them, y is the predicted value, x i is the eigenvalue, β i is the regression coefficient.
[0037] (2) Use decision tree method to build data center power prediction model;
[0038] It is suitable for data with strong nonlinear relationships. It builds a tree structure by recursively splitting the feature space. The model form is:
[0039]
[0040] Where pi is the probability of the i-th class.
[0041] (3) Use the random forest method to build a data center power prediction model;
[0042] By integrating multiple decision trees, the stability and accuracy of the model are improved. The model form is:
[0043]
[0044] Where T is the number of decision trees and ht(x) is the predicted value of the tth tree.
[0045] (4) A data center power prediction model is constructed using a neural network method. The model expression is:
[0046] Simulate complex nonlinear relationships through multiple layers of neurons;
[0047] y=f(W x +b)
[0048] Among them, W is the weight matrix, b is the bias vector, and f is the activation function, such as ReLU, sigmoid, etc.
[0049] (5) Using the gradient descent method to train the various machine learning models constructed above;
[0050] The gradient descent method is used to iteratively update the model parameters to minimize the loss function:
[0051]
[0052] Among them, θ is the model parameter, α is the learning rate, is the gradient of the loss function.
[0053] (6) The optimal model is selected through cross-validation as the data center power prediction model constructed using the neural network method.
[0054] Moreover, the specific method of step 5 is:
[0055] The mean square error (MSE) is used to verify the performance of the data center power prediction model built by the neural network method:
[0056] The mean squared error is the average of the squares of the differences between the predicted values and the true values, and the formula is:
[0057]
[0058] Among them, yi is the true value, is the predicted value and n is the number of samples.
[0059] Moreover, the specific steps of step 6 include:
[0060] (1) Model saving: Save the trained model as a file. Common formats include pickle and joblib.
[0061] (2) Model loading: Load the saved model in actual application and make predictions.
[0062] (3) Real-time prediction: Obtain new data and make predictions in real time through API interfaces or scheduled tasks.
[0063] (4) Result presentation: The prediction results are presented to users, such as through charts, reports, etc.
[0064] Moreover, after step 6, the method further includes the following steps:
[0065] Step 7: Continuously optimize the trained data center power prediction model constructed by the neural network method;
[0066] The specific steps of step 7 include:
[0067] (1) Update data regularly: Regularly collect new data, update the training set, and keep the model up to date.
[0068] (2) Model tuning: Adjust model parameters based on new data and feedback to improve prediction accuracy.
[0069] (3) Performance monitoring: Regularly monitor the predictive performance of the model to identify and resolve problems in a timely manner.
[0070] (4) User feedback: Collect user feedback and continuously improve the model.
[0071] Advantages and beneficial effects of the present invention:
[0072] This paper proposes a data center power forecasting method based on machine learning algorithm, including six steps: data collection, data preprocessing, feature engineering, model selection and training, model evaluation and deployment application; this paper provides a scientific basis for data center energy management by building an efficient forecasting model. Experimental results show that the forecasting model based on neural network performs well in accuracy and stability, and can effectively reduce the energy consumption of data centers and improve operational efficiency. In the future, we will continue to optimize the model, expand application scenarios, and provide advanced power forecasting solutions for more data centers. DETAILED DESCRIPTION
[0073] The following further describes the embodiments of the present invention:
[0074] A data center power prediction method based on a machine learning algorithm comprises the following steps:
[0075] Step 1: Obtain data from various monitoring devices in the data center;
[0076] Data collection is the basis of electricity forecasting, and high-quality data is the prerequisite for building an accurate forecasting model.
[0077] The step 1 obtains the following types of data from various monitoring devices in the data center:
[0078] (1) Historical electricity consumption data: including total electricity consumption in each time period, electricity consumption of each device, etc.
[0079] (2) Equipment operation status data: including the operation status information of servers, storage devices, network devices, etc., such as CPU utilization, memory usage, disk read and write rates, etc.
[0080] (3) Environmental data: including environmental parameters such as temperature, humidity, and light intensity inside and outside the data center.
[0081] (4) External influencing factors data: including weather forecast, holiday information, user visits and other external influencing factors.
[0082] Step 2: preprocess the data obtained in step 1;
[0083] Data preprocessing is a key step to ensure data quality;
[0084] The specific steps of step 2 include:
[0085] (1) Data cleaning: remove outliers and noise from the data and fill in missing values. Common filling methods include forward filling, backward filling, mean filling, etc.
[0086] The main purpose of data cleaning is to remove outliers and fill missing values. Outliers can be detected by statistical methods, such as the Z-Score method or the IQR method. Missing values can be filled by interpolation methods, such as linear interpolation or polynomial interpolation.
[0087] (2) Data standardization: Convert the data to the same scale. Commonly used methods include Min-Max Scaling and Z-Score standardization.
[0088] Data standardization is to eliminate the impact of different dimensions between features and make model training more stable. Common standardization methods are:
[0089]
[0090] Among them, x' is the standardized value, x is the original value, min(x) and max(x) are the minimum and maximum values of the feature, respectively, and μ and σ are the mean and standard deviation of the feature, respectively.
[0091] (3) Data normalization: Convert the data to values between 0 and 1. Common methods include MinMaxScaler.
[0092] Step 3: Based on the data preprocessed in step 2, perform feature engineering and extract features;
[0093] Feature engineering is a key step to improve the prediction performance of models, which mainly includes feature selection, feature construction and feature transformation.
[0094] The specific steps of step 3 include:
[0095] (1) Feature selection:
[0096] Using the filtering method, the historical power consumption of the data center during the flat, peak and valley periods is selected as the features;
[0097] The purpose of feature selection is to select the most relevant features from a large number of features, reduce model complexity and improve model performance.
[0098] (2) Feature structure:
[0099] The time feature method is used and the ISO 8601 format is adopted as the timestamp of the practice feature. The timestamp is divided into quarters. At the same time, the aggregation feature method is used to aggregate the quarters and the peak, valley and normal period electricity in the time feature to calculate the average, maximum and minimum values of the peak, valley and normal period electricity in different quarters.
[0100] Feature construction refers to deriving new features from existing features to provide more useful information.
[0101] (3) Feature conversion:
[0102] The eigenvalues are transformed into polynomials using polynomial transformation.
[0103] In order to improve the performance of the model, the completed time characteristics are transformed. The present invention adopts polynomial transformation to perform polynomial transformation on the characteristic values to increase the nonlinear ability of the model.
[0104] Step 4: Based on the features extracted in step 3, multiple machine learning algorithms are used to build and train a data center power prediction model, and the optimal model is selected through cross-validation;
[0105] The specific steps of using multiple machine learning algorithms to construct and train a data center power prediction model in step 4 include:
[0106] Model selection and training are the core steps in building a prediction model. The present invention uses a variety of machine learning algorithms for model training and selects the optimal model through cross-validation.
[0107] (1) Use linear regression method to build a data center power prediction model;
[0108] Applicable to data with strong linear relationship, the model form is:
[0109] y=β0+β1x1+β2x2+…+β n x n
[0110] Among them, y is the predicted value, x i is the eigenvalue, β i is the regression coefficient.
[0111] (2) Use decision tree method to build data center power prediction model;
[0112] It is suitable for data with strong nonlinear relationships. It builds a tree structure by recursively splitting the feature space. The model form is:
[0113]
[0114] Where pi is the probability of the i-th class.
[0115] (3) Use the random forest method to build a data center power prediction model;
[0116] By integrating multiple decision trees, the stability and accuracy of the model are improved. The model form is:
[0117]
[0118] Where T is the number of decision trees and ht(x) is the predicted value of the tth tree.
[0119] (4) A data center power prediction model is constructed using a neural network method. The model expression is:
[0120] Complex nonlinear relationships are simulated through multiple layers of neurons.
[0121] y=f(W x +b)
[0122] Among them, W is the weight matrix, b is the bias vector, and f is the activation function, such as ReLU, sigmoid, etc.
[0123] (5) Using the gradient descent method to train the various machine learning models constructed above;
[0124] In this embodiment, the gradient descent method is used to iteratively update the model parameters to minimize the loss function:
[0125]
[0126] Among them, θ is the model parameter, α is the learning rate, is the gradient of the loss function.
[0127] The purpose of model training is to learn model parameters through historical data so that the model can accurately predict future data.
[0128] The optimal model selected by cross-validation in step 4 is a data center power prediction model constructed by a neural network method.
[0129] After verification, the data center power prediction model constructed by the neural network method has the highest accuracy in the test results of the test set, so the neural network method is used to construct the data center power prediction model.
[0130] Step 5: Evaluate the data center power prediction model constructed by the neural network method;
[0131] Model evaluation is an important step to verify model performance.
[0132] The specific method of step 5 is:
[0133] The mean square error (MSE) is used to verify the performance of the data center power prediction model built by the neural network method:
[0134] The mean squared error is the average of the squares of the differences between the predicted values and the true values, and the formula is:
[0135]
[0136] Among them, yi is the true value, is the predicted value and n is the number of samples.
[0137] Step 6: Apply the trained model to actual scenarios to achieve automated prediction and obtain data center power prediction results.
[0138] The specific steps of step 6 include:
[0139] (1) Model saving: Save the trained model as a file. Common formats include pickle and joblib.
[0140] (2) Model loading: Load the saved model in actual application and make predictions.
[0141] (3) Real-time prediction: Obtain new data and make predictions in real time through API interfaces or scheduled tasks.
[0142] (4) Result presentation: The prediction results are presented to users, such as through charts, reports, etc.
[0143] Step 7: Continuously optimize the trained data center power prediction model constructed by the neural network method;
[0144] Continuous optimization of the model is an important part of ensuring predictive performance.
[0145] The specific steps of step 7 include:
[0146] (1) Update data regularly: Regularly collect new data, update the training set, and keep the model up to date.
[0147] (2) Model tuning: Adjust model parameters based on new data and feedback to improve prediction accuracy.
[0148] (3) Performance monitoring: Regularly monitor the predictive performance of the model to identify and resolve problems in a timely manner.
[0149] (4) User feedback: Collect user feedback and continuously improve the model.
[0150] In order to verify the effectiveness of the method of the present invention, we conducted experiments in the data center. The experimental data includes historical electricity consumption data, equipment operation status data, environmental data and external influencing factor data in the past year. We used multiple models such as linear regression, decision tree, random forest, support vector machine and neural network for comparative experiments.
[0151] 1) Experimental setup
[0152] Dataset: Use historical data from the past year, divided into training set and test set, with a ratio of 7:3.
[0153] Evaluation indicators: Mean square error (MSE), mean absolute error (MAE) and coefficient of determination (R 2 ) for evaluation.
[0154] Experimental environment: Use Python programming language and scikit-learn library for model training and evaluation.
[0155] 2) Result analysis
[0156] Model performance: Neural network model performance in MSE, MAE and R 2 The excellent performance in all three indicators shows that it can better capture the complex patterns of power changes in data centers.
[0157] Model stability: The performance of the random forest model on different data sets is relatively stable and has good generalization ability.
[0158] Computational efficiency: The linear regression model has the highest computational efficiency and is suitable for real-time prediction scenarios.
[0159] The working principle of the present invention is:
[0160] In recent years, with the development of machine learning technology, more and more studies have shown that prediction methods based on machine learning can significantly improve the accuracy and reliability of predictions. Machine learning algorithms can extract features from a large amount of historical data, discover potential patterns and laws, and thus provide more accurate prediction results. In addition, machine learning models also have strong adaptability and flexibility, and can maintain good prediction performance in a constantly changing environment.
[0161] Therefore, the present invention aims to study and implement a data center power prediction method based on machine learning algorithm, and provide a scientific basis for the energy management of the data center by constructing an efficient prediction model, so as to achieve the goal of energy conservation and emission reduction and improve the operational efficiency of the data center.
[0162] The method of the present invention has been put into practical application in a large data center. By deploying a power prediction model based on a neural network, the data center can predict future power demand in advance, so as to take corresponding measures, such as adjusting the operating status of the equipment, optimizing the cooling system, etc., effectively reducing energy consumption and improving operational efficiency. By implementing the method of the present invention, the energy consumption of the data center has been reduced by 15%, saving about 1 million yuan in electricity bills each year. At the same time, the PUE (Power Usage Effectiveness) of the data center has dropped from 1.6 to 1.4, reaching the industry-leading level. The introduction of the prediction model makes the operation of the data center more intelligent. Managers can formulate maintenance plans in advance based on the prediction results, reduce the occurrence of sudden failures, and improve the reliability and stability of the system.
[0163] It should be emphasized that the embodiments of the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific implementation modes. Any other implementation modes derived by those skilled in the art based on the technical solutions of the present invention also fall within the scope of protection of the present invention.
Claims
1. A data center power prediction method based on machine learning algorithm, characterized by: The following steps are involved: Step 1: Obtain data from various monitoring devices in the data center; Step 2: preprocess the data obtained in step 1; Step 3: Based on the data preprocessed in step 2, perform feature engineering and extract features; Step 4: Based on the features extracted in step 3, multiple machine learning algorithms are used to build and train a data center power prediction model, and the optimal model is selected through cross-validation; Step 5: Evaluate the data center power prediction model constructed by the neural network method; Step 6: Apply the trained model to actual scenarios to achieve automated prediction and obtain data center power prediction results.
2. The data center power prediction method based on machine learning algorithm according to claim 1 is characterized by: The step 1 obtains the following types of data from various monitoring devices in the data center: (1) Historical electricity consumption data: including total electricity consumption in each time period and electricity consumption of each device; (2) Equipment operation status data: including operation status information of servers, storage devices, and network devices; (3) Environmental data: including temperature, humidity, and light intensity environmental parameters inside and outside the data center; (4) External influencing factors data: including weather forecast, holiday information, and user visit volume.
3. The data center power prediction method based on machine learning algorithm according to claim 1 is characterized by: The specific steps of step 2 include: (1) Data cleaning: remove outliers and noise from the data and fill in missing values; (2) Data standardization: converting data to the same scale; (3) Data normalization: convert the data into values between 0 and 1.
4. The data center power prediction method based on machine learning algorithm according to claim 1 is characterized by: The specific steps of step 3 include: (1) Feature selection: Using the filtering method, the historical power consumption of the data center during the flat, peak and valley periods is selected as the features; (2) Feature structure: Using the time feature method, the ISO 8601 format is used as the timestamp of the practice feature, and the timestamp is divided into quarters. At the same time, the aggregation feature method is used to aggregate the quarters and the peak, valley and normal period electricity in the time feature, and the average, maximum and minimum values of the peak, valley and normal period electricity in different quarters are calculated; (3) Feature conversion: The eigenvalues are transformed into polynomials using polynomial transformation.
5. The data center power prediction method based on machine learning algorithm according to claim 1 is characterized by: The specific steps of using multiple machine learning algorithms to construct and train a data center power prediction model in step 4 include: (1) Use linear regression method to build a data center power prediction model; Applicable to data with strong linear relationship, the model form is: y=β0+β1x1+β2x2+…+β n x n Among them, y is the predicted value, x i is the eigenvalue, β i is the regression coefficient; (2) Use decision tree method to build data center power prediction model; It is suitable for data with strong nonlinear relationships. It builds a tree structure by recursively splitting the feature space. The model form is: Among them, pi is the probability of the i-th class; (3) Use the random forest method to build a data center power prediction model; By integrating multiple decision trees, the stability and accuracy of the model are improved. The model form is: Where T is the number of decision trees, and ht(x) is the predicted value of the tth tree; (4) A data center power prediction model is constructed using a neural network method. The model expression is: Simulate complex nonlinear relationships through multiple layers of neurons; y=f(W x +b) Among them, W is the weight matrix, b is the bias vector, and f is the activation function, such as ReLU, sigmoid, etc.; (5) Using the gradient descent method to train the various machine learning models constructed above; The gradient descent method is used to iteratively update the model parameters to minimize the loss function: Among them, θ is the model parameter, α is the learning rate, is the gradient of the loss function; (6) The optimal model is selected through cross-validation as the data center power prediction model constructed using the neural network method.
6. The data center power prediction method based on machine learning algorithm according to claim 1 is characterized by: The specific method of step 5 is: The mean square error (MSE) is used to verify the performance of the data center power prediction model built by the neural network method: The mean squared error is the average of the squares of the differences between the predicted values and the true values, and the formula is: Among them, yi is the true value, is the predicted value and n is the number of samples.
7. The data center power prediction method based on machine learning algorithm according to claim 1 is characterized by: The specific steps of step 6 include: (1) Model saving: Save the trained model as a file. Common formats include pickle and joblib. (2) Model loading: Load the saved model in actual application and make predictions; (3) Real-time prediction: Obtain new data and make predictions in real time through API interfaces or scheduled tasks; (4) Result presentation: The prediction results are presented to users, such as through charts, reports, etc.
8. The data center power prediction method based on machine learning algorithm according to claim 1 is characterized by: After step 6, the method further includes the following steps: Step 7: Continuously optimize the trained data center power prediction model constructed by the neural network method; The specific steps of step 7 include: (1) Update data regularly: regularly collect new data, update the training set, and keep the model up to date; (2) Model tuning: Adjust model parameters based on new data and feedback to improve prediction accuracy; (3) Performance monitoring: Regularly monitor the prediction performance of the model to detect and resolve problems in a timely manner; (4) User feedback: Collect user feedback and continuously improve the model.
Citation Information
Cited By
Bed height detection method based on machine learning
CN120742815A