Resource allocation method and device for virtual machine

Abstract

The invention discloses a resource allocation method and device for a virtual machine, and the method comprises the steps: carrying out the calculation of a linear coefficient between every two pieces of data in real-time load, performance indexes and energy consumption collected from a target virtual machine, generating three sub-data sets, integrating the three sub-data sets, and outputting an allocation data set; the distribution data set is averagely divided into a sample data set and a training data set, the training data set is used for training the initial machine model, the sample data set, the target energy cost and the target performance index are input into the trained model for prediction, and resources are distributed according to resource demand prediction output by the model. The incidence relation among various parameters of the virtual machine is defined by calculating the linear coefficients among the three types of data, and the prediction accuracy and precision of the prediction model are improved; and prediction and allocation are performed according to the data acquired in real time, so that dynamic adjustment of a virtual machine allocation strategy is realized, and the practicability of resource allocation and the sufficiency of resource utilization are improved.

Description

Technical Field

[0001] The present invention relates to the field of network technology, and in particular to a resource allocation method and device for a virtual machine. Background Art

[0002] In traditional virtualization environments, resource allocation and management are usually static and rule-based. System administrators manually configure the allocation ratio of virtual machines and physical resources according to pre-set rules and policies. However, this static resource allocation method generally has the following problems: first, low resource utilization: due to static allocation, resources are often not fully utilized; second, it is difficult to cope with dynamic workloads: traditional resource allocation methods cannot dynamically adjust resource allocation according to changing workloads; and then there is energy waste: static resource allocation methods cannot consider factors such as energy costs and environmental sustainability.

[0003] In order to solve the above problems, the existing technology has proposed the following solutions, including: rule-based resource management solution: using pre-defined rules and policies to statically allocate physical resources to virtual machines; load balancing algorithm: by monitoring the server load and migrating the virtual machine to a server with lower load according to the load situation; energy management tool: energy management tool helps system administrators monitor and control server energy consumption.

[0004] However, the above solutions are limited to solving some resource allocation problems of virtual machines, but ignore other resource allocation problems of virtual machines, resulting in that the above solutions cannot achieve more accurate and dynamic resource allocation. Summary of the invention

[0005] The present invention provides a method and device for allocating resources for a virtual machine, so as to achieve the technical effect of improving the accuracy and dynamic flexibility of resource allocation.

[0006] In order to solve the above technical problem, an embodiment of the present invention provides a resource allocation method for a virtual machine, comprising the following steps:

[0007] Acquire the real-time load, performance index and energy consumption of the target virtual machine in real time, calculate the linear coefficient between each two data of the real-time load, the energy consumption and the performance index, output the corresponding three sub-data sets, and then merge the three sub-data sets and output them as the allocation data set;

[0008] The allocated data set is evenly divided into a sample data set and a training data set, the training data set is input into a preset initial machine model for training, and after the training is completed, the sample data set, the target energy cost and the target performance index are input into the trained resource demand prediction model for prediction and output of the resource demand prediction of the target virtual machine, and then the virtual machine is classified according to the resource demand prediction.

[0009] The resource allocation method provided by the present invention first collects various operating parameters of the target virtual machine in real time, including parameter data such as real-time load, performance index and energy consumption, so as to realize real-time monitoring of the operating status of the virtual machine. At the same time, the collected data is used as training and sample data of the subsequent prediction model to improve the practicality and dynamic flexibility of the prediction data. The linear coefficient between each two data in the real-time load data, energy consumption data and performance index data is calculated respectively, and three sub-data sets are generated according to the calculated linear coefficients as the training and input data of the prediction model, so that the prediction model fully considers and clarifies the correlation between various parameters in the virtual machine during prediction, further improving the prediction accuracy and precision of the prediction model.

[0010] The allocation data set is evenly divided into a sample data set and a training data set, which improves the sensitivity of the trained model to the sample data set. The target energy cost and target performance index are input into the resource demand prediction model together with the sample data set for prediction, so that the output prediction data not only considers the energy efficiency and energy sustainable development issues, but also takes into account the energy cost and virtual machine performance requirements, and realizes the management and optimization of energy in the resource demand prediction output by the model, which improves resource utilization while reducing energy consumption and adverse effects on the environment. Prediction and allocation based on real-time collected data also realizes the dynamic adjustment of virtual machine allocation strategy, which improves the practicality of resource allocation and the adequacy of resource utilization.

[0011] As a preferred example, the calculating of the linear coefficient between each two data of the real-time load, the energy consumption and the performance index, outputting the corresponding three sub-data sets, and then merging the three sub-data sets and outputting them as the allocation data set specifically includes:

[0012] Labeling the real-time load, the energy consumption and the performance index according to the collection time points respectively, and associating and combining the real-time load, the energy consumption and the performance index having the same collection time points after labeling;

[0013] Perform linear regression analysis on the associated combined data according to the collection time points, output corresponding linear coefficients, and then associate the linear coefficients about the real-time load and the energy consumption among the linear coefficients with the real-time load and the energy consumption as a first sub-dataset, associate the linear coefficients about the real-time load and the performance index among the linear coefficients with the real-time load and the performance index as a second sub-dataset, and associate the linear coefficients about the energy consumption and the performance index among the linear coefficients with the energy consumption and the performance index as a third sub-dataset.

[0014] According to the time point when the system collects each real-time load data, energy consumption data and performance index data, all the collected data are labeled and divided, so that the data of the three virtual machine operating parameters corresponding to each collection time point can be obtained. After the three types of data at each time point are associated and combined, the combined data are linearly regressed according to the collection time point, so that the influence of the three types of data in the combined data on each other over time can be obtained. By combining the partial data on load and energy, partial data on load and performance, and partial data on energy and performance in the obtained data, three corresponding allocation data sets can be output. Since the output allocation data set includes not only the operating parameter data of the target virtual machine collected by the system in real time, but also the influence relationship between various parameters, the complex relationship between factors such as load, performance and energy cost is comprehensively considered, which further improves the utilization and accuracy of system resource allocation.

[0015] As a preferred example, the training data set is input into a preset initial machine model for training, specifically:

[0016] Performing outlier removal, missing value supplementation and data normalization on the training data set in turn, and outputting corresponding training sample data;

[0017] Extracting features from the training sample data and the attribute data of the target virtual machine respectively, outputting corresponding feature subsets, and calling an importance analysis algorithm, a correlation analysis algorithm, and a recursive feature elimination algorithm to sequentially screen and process the feature subsets, and outputting feature data;

[0018] The training sample data is evenly divided into a test set and a training set, the feature data and the training set are input into the initial machine model for training, and the model is tested with the test set after training, and the hyperparameters of the model are adjusted by the loss function until the data output by the model test satisfies.

[0019] The accuracy and completeness of the training sample data are improved by removing outliers and supplementing missing values ​​in the training data set, while normalizing the data avoids the impact of scale differences between different feature subsets extracted by the system during feature extraction on model training. When extracting features from the training sample data, the system also adds the attribute data of the target virtual machine during the extraction process, which not only enriches the feature extraction samples, but also further improves the prediction ability of the model by extracting diverse feature types.

[0020] By further screening the feature subsets through importance analysis algorithm, correlation analysis algorithm and recursive feature elimination algorithm, the feature subsets that have the greatest influence on model prediction can be accurately located, thereby achieving prediction optimization of the resource demand prediction model and improving the prediction accuracy and prediction effect of the model.

[0021] As a preferred example, the feature data and the training set are input into the initial machine model for training, and the model is tested by the test set after training, and the hyperparameters of the model are adjusted by the loss function until the data output by the model test satisfies, specifically:

[0022] Input the feature data and the training set into the initial machine model for training, input the test set into the trained model to test the model, output first test data, and calculate the root mean square error and mean absolute error between the first test data and first real data as the loss function;

[0023] The learning rate, regularization parameter and number of iterations of the model are adjusted according to the calculated loss function, and the model iteration is trained, tested and hyperparameters adjusted until the loss function calculated and output by the test model is less than the preset loss function threshold. The model corresponding to the loss function is then output as a resource demand prediction model.

[0024] In order to improve the prediction accuracy and prediction effect of the model, the resource allocation method provided by the present invention uses root mean square error and mean absolute error as loss functions to evaluate the prediction performance of the model. If the evaluation finds that the prediction error is large, the hyperparameters of the model are adjusted according to the calculated loss function, including hyperparameters such as the model's learning rate, regularization parameter and number of iterations.

[0025] By repeatedly iterating the above-mentioned training, testing, and parameter adjustment processes, the system can select a set of hyperparameters with the best performance based on the loss function calculated each time. After determination, the model corresponding to this set of parameters is determined as the resource demand prediction model, so that both the sample data set and the training data set perform well in this model, thereby improving the prediction effect of the model.

[0026] As a preferred example, classifying the resources of the virtual machines according to the resource demand prediction specifically includes:

[0027] Generating a corresponding first resource allocation strategy according to the resource demand prediction, and allocating physical resources of the target virtual machine according to the first resource allocation strategy;

[0028] The energy consumption and the performance indicators of the target virtual machine are collected in real time, and the collected energy consumption and the performance indicators are input into the resource demand prediction model for prediction, and a corresponding second resource allocation strategy is generated according to the prediction data output by the model prediction, and the first resource allocation strategy is adjusted according to the second resource allocation strategy, and then the physical resources are allocated according to the adjusted strategy.

[0029] In order to achieve dynamic adjustment of resource allocation strategies, improve the degree of adaptation between resource allocation strategies and the actual situation of virtual machines, and reduce the energy consumption of virtual machines, the resource allocation method provided by the present invention will continuously collect energy consumption data and indicator data of virtual machines in real time after allocating resources to virtual machines according to the first allocation strategy, so that the prediction model can re-predict and output prediction data and generate a second resource allocation strategy, and then adjust the first strategy according to the newly generated second resource allocation strategy, so that the adjusted strategy is more in line with the current operating status and energy consumption of the virtual machine and the performance requirements of different virtual machines, thereby achieving dynamic optimization and adjustment of resource allocation strategies, further reducing the energy consumption and cost consumption of virtual machines, and building a green and low-carbon data center.

[0030] Accordingly, the present invention also provides a resource allocation device for a virtual machine, the resource allocation device comprising a data acquisition processing module and a model training prediction module;

[0031] The data acquisition and processing module is used to acquire the real-time load, performance index and energy consumption of the target virtual machine in real time, calculate the linear coefficient between each two data in the real-time load, the energy consumption and the performance index, output the corresponding three sub-data sets, and then merge the three sub-data sets and output them as the allocation data set;

[0032] The model training prediction module is used to evenly divide the allocated data set into a sample data set and a training data set, input the training data set into a preset initial machine model for training, and after the training is completed, input the sample data set, target energy cost and target performance index into the trained resource demand prediction model, so that the resource demand prediction model outputs the resource demand prediction of the target virtual machine, and classifies the resources of the virtual machine according to the resource demand prediction.

[0033] As a preferred example, the data acquisition and processing module calculates the linear coefficient between each two data of the real-time load, the energy consumption and the performance index, outputs the corresponding three sub-data sets, and then merges the three sub-data sets and outputs them as a distribution data set, specifically including:

[0034] Labeling the real-time load, the energy consumption and the performance index according to the collection time points respectively, and associating and combining the real-time load, the energy consumption and the performance index having the same collection time points after labeling;

[0035] Perform linear regression analysis on the associated combined data according to the collection time points, output corresponding linear coefficients, and then associate the linear coefficients about the real-time load and the energy consumption among the linear coefficients with the real-time load and the energy consumption as a first sub-dataset, associate the linear coefficients about the real-time load and the performance index among the linear coefficients with the real-time load and the performance index as a second sub-dataset, and associate the linear coefficients about the energy consumption and the performance index among the linear coefficients with the energy consumption and the performance index as a third sub-dataset.

[0036] As a preferred example, the model training prediction module inputs the training data set into a preset initial machine model for training, specifically:

[0037] Performing outlier removal, missing value supplementation and data normalization on the training data set in turn, and outputting corresponding training sample data;

[0038] Extracting features from the training sample data and the attribute data of the target virtual machine respectively, outputting corresponding feature subsets, and calling an importance analysis algorithm, a correlation analysis algorithm, and a recursive feature elimination algorithm to sequentially screen and process the feature subsets, and outputting feature data;

[0039] The training sample data is evenly divided into a test set and a training set, the feature data and the training set are input into the initial machine model for training, and the model is tested with the test set after training, and the hyperparameters of the model are adjusted by the loss function until the data output by the model test satisfies.

[0040] As a preferred example, the model training prediction module inputs the feature data and the training set into the initial machine model for training, and tests the model through the test set after training, and adjusts the hyperparameters of the model through the loss function until the data output by the model test satisfies, specifically:

[0041] Input the feature data and the training set into the initial machine model for training, input the test set into the trained model to test the model, output first test data, and calculate the root mean square error and mean absolute error between the first test data and first real data as the loss function;

[0042] The learning rate, regularization parameter and number of iterations of the model are adjusted according to the calculated loss function, and the model iteration is trained, tested and hyperparameters adjusted until the loss function calculated and output by the test model is less than the preset loss function threshold. The model corresponding to the loss function is then output as a resource demand prediction model.

[0043] As a preferred example, the model training prediction module classifies resources of the virtual machine according to the resource demand prediction, specifically including:

[0044] Generating a corresponding first resource allocation strategy according to the resource demand prediction, and allocating physical resources of the target virtual machine according to the first resource allocation strategy;

[0045] The consumption data and indicator data of the target virtual machine are collected in real time, and the collected consumption data and indicator data are input into the resource demand prediction model for prediction, and a corresponding second resource allocation strategy is generated according to the prediction data output by the model prediction, and the first resource allocation strategy is adjusted according to the second resource allocation strategy, and then the physical resources are allocated according to the adjusted strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 : A flow chart of an embodiment of a resource allocation method for a virtual machine provided by the present invention;

[0047] Figure 2 : A structural diagram of an embodiment of a resource allocation device for a virtual machine provided by the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] Embodiment 1

[0050] Please refer to Figure 1, is a flow chart of an embodiment of a resource allocation method for a virtual machine provided by the present invention, including steps 101 to 102, each of which is specifically as follows:

[0051] Step 101: collect and acquire the real-time load, performance index and energy consumption of the target virtual machine in real time, calculate the linear coefficient between each two data in the real-time load, the energy consumption and the performance index, output the corresponding three sub-data sets, and then merge the three sub-data sets and output them as the allocation data set.

[0052] The resource allocation method provided by the embodiment of the present invention first collects various operating parameters of the target virtual machine in real time, including parameter data such as real-time load, performance index and energy consumption, so as to realize real-time monitoring of the operating status of the virtual machine. At the same time, the collected data is used as training and sample data of the subsequent prediction model to improve the practicality and dynamic flexibility of the prediction data. The linear coefficient between each two data in the real-time load data, energy consumption data and performance index data is calculated respectively, and three sub-data sets are generated according to the calculated linear coefficients as the training and input data of the prediction model, so that the prediction model fully considers and clarifies the correlation between various parameters in the virtual machine during prediction, further improving the prediction accuracy and precision of the prediction model.

[0053] In this embodiment, when the system collects data on the target virtual machine in real time, the collected data includes not only the server load set, i.e., the real-time load, the virtual machine performance indicators, i.e., the performance indicators and energy consumption, all of which are real-time data in the virtualization environment, but also the virtual machine's CPU utilization, memory usage, network traffic and other indicators, as well as related resource requirements, such as the number of CPU cores, memory capacity and other data, so as to enrich the training data of the model.

[0054] Specifically, the present embodiment calculates the linear coefficient between each two data of the real-time load, the energy consumption and the performance index, outputs three corresponding sub-data sets, and then merges the three sub-data sets and outputs them as a distribution data set, specifically including:

[0055] Labeling the real-time load, the energy consumption and the performance index according to the collection time points respectively, and associating and combining the real-time load, the energy consumption and the performance index having the same collection time points after labeling;

[0056] Perform linear regression analysis on the associated combined data according to the collection time points, output corresponding linear coefficients, and then associate the linear coefficients about the real-time load and the energy consumption among the linear coefficients with the real-time load and the energy consumption as a first sub-dataset, associate the linear coefficients about the real-time load and the performance index among the linear coefficients with the real-time load and the performance index as a second sub-dataset, and associate the linear coefficients about the energy consumption and the performance index among the linear coefficients with the energy consumption and the performance index as a third sub-dataset.

[0057] According to the time point when the system collects each real-time load data, energy consumption data and performance index data, all the collected data are labeled and divided, so that the data of the three virtual machine operating parameters corresponding to each collection time point can be obtained. After the three types of data at each time point are associated and combined, the combined data are linearly regressed according to the collection time point, so that the influence of the three types of data in the combined data on each other over time can be obtained. By combining the partial data on load and energy, partial data on load and performance, and partial data on energy and performance in the obtained data, three corresponding allocation data sets can be output. Since the output allocation data set includes not only the operating parameter data of the target virtual machine collected by the system in real time, but also the influence relationship between various parameters, the complex relationship between factors such as load, performance and energy cost is comprehensively considered, which further improves the utilization and accuracy of system resource allocation.

[0058] In order to facilitate the subsequent prediction model to learn the complex relationship between load, performance and energy, the system first calculates the linear coefficients between the various types of collected data before inputting the collected real-time data into the initial machine model training. That is, the influence relationship between the various parameters is clarified through the calculated linear coefficients, and the calculated linear coefficients are added to the allocated data set for the prediction model to carry out subsequent training and prediction, thereby improving the training efficiency of the model.

[0059] Step 102: Divide the allocated data set evenly into a sample data set and a training data set, input the training data set into a preset initial machine model for training, and after the training is completed, input the sample data set, target energy cost and target performance index into the trained resource demand prediction model for prediction and output of the resource demand prediction of the target virtual machine, and then classify the virtual machine according to the resource demand prediction.

[0060] The allocation data set is evenly divided into a sample data set and a training data set, which improves the sensitivity of the trained model to the sample data set. The target energy cost and target performance index are input into the resource demand prediction model together with the sample data set for prediction, so that the output prediction data not only considers the energy efficiency and energy sustainable development issues, but also takes into account the energy cost and virtual machine performance requirements, and realizes the management and optimization of energy in the resource demand prediction output by the model, which improves resource utilization while reducing energy consumption and adverse effects on the environment. Prediction and allocation based on real-time collected data also realizes the dynamic adjustment of virtual machine allocation strategy, which improves the practicality of resource allocation and the adequacy of resource utilization.

[0061] Specifically, in this embodiment, the training data set is input into a preset initial machine model for training, specifically:

[0062] Performing outlier removal, missing value supplementation and data normalization on the training data set in turn, and outputting corresponding training sample data;

[0063] Extracting features from the training sample data and the attribute data of the target virtual machine respectively, outputting corresponding feature subsets, and calling an importance analysis algorithm, a correlation analysis algorithm, and a recursive feature elimination algorithm to sequentially screen and process the feature subsets, and outputting feature data;

[0064] The training sample data is evenly divided into a test set and a training set, the feature data and the training set are input into the initial machine model for training, and the model is tested with the test set after training, and the hyperparameters of the model are adjusted by the loss function until the data output by the model test satisfies.

[0065] The accuracy and completeness of the training sample data are improved by removing outliers and supplementing missing values ​​in the training data set, while normalizing the data avoids the impact of scale differences between different feature subsets extracted by the system during feature extraction on model training. When extracting features from the training sample data, the system also adds the attribute data of the target virtual machine during the extraction process, which not only enriches the feature extraction samples, but also further improves the prediction ability of the model by extracting diverse feature types.

[0066] By further screening the feature subsets through importance analysis algorithm, correlation analysis algorithm and recursive feature elimination algorithm, the feature subsets that have the greatest influence on model prediction can be accurately located, thereby achieving prediction optimization of the resource demand prediction model and improving the prediction accuracy and prediction effect of the model.

[0067] In this embodiment, when the system performs data preprocessing on the training data set, it first performs data cleaning on the data, including outlier removal and missing value supplementation, so as to check whether there are missing values ​​or outliers in the training data set. If there are, it will be processed accordingly to avoid the missing values ​​or outliers from affecting the model prediction. Then, the data is normalized and scaled to a smaller range to avoid the scale differences between different features extracted from the data from affecting the model training. For example, the data can be scaled to between 0 and 1 through Min-Max normalization.

[0068] After completing data preprocessing, the system can perform feature engineering on the data, that is, feature subset extraction. In feature engineering construction, the system not only extracts features from the training data set, but also extracts features from the attribute data of the target virtual machine, that is, attribute parameters. In the intelligent allocation of resources in a virtualized environment, in addition to real-time data and historical data, incorporating diverse feature types such as virtual machine configuration information, network traffic characteristics, disk I / O characteristics, virtual machine operating status characteristics, and operating system indicator characteristics will further enhance the model's predictive capabilities. The above-mentioned attribute parameter features provide rich information about virtual machine performance, resource requirements, and operating status, which helps the model to more comprehensively understand the characteristics and changes of the virtualized environment. For example, calculating the rate of change of a virtual machine's average CPU utilization or memory usage over the past hour as a feature can provide more information about resource requirements.

[0069] By comprehensively considering the several feature subsets obtained by extracting the above data and combining them with the corresponding feature selection algorithm, the system can accurately screen out the feature subsets that have the greatest impact on model prediction, thereby optimizing the accuracy and prediction effect of the resource demand prediction model. This embodiment does not further limit the feature training algorithm. This embodiment preferably uses a feature importance analysis algorithm, a feature correlation analysis algorithm, and a recursive feature elimination algorithm.

[0070] Furthermore, in this embodiment, the feature data and the training set are input into the initial machine model for training, and the model is tested by the test set after training, and the hyperparameters of the model are adjusted by the loss function until the data output by the model test satisfies, specifically:

[0071] Input the feature data and the training set into the initial machine model for training, input the test set into the trained model to test the model, output first test data, and calculate the root mean square error and mean absolute error between the first test data and first real data as the loss function;

[0072] The learning rate, regularization parameter and number of iterations of the model are adjusted according to the calculated loss function, and the model iteration is trained, tested and hyperparameters adjusted until the loss function calculated and output by the test model is less than the preset loss function threshold. The model corresponding to the loss function is then output as a resource demand prediction model.

[0073] In order to improve the prediction accuracy and prediction effect of the model, the resource allocation method provided in the embodiment of the present invention uses root mean square error and mean absolute error as loss functions to evaluate the prediction performance of the model. If the evaluation finds that the prediction error is large, the hyperparameters of the model are adjusted according to the calculated loss function, including hyperparameters such as the model's learning rate, regularization parameter, and number of iterations.

[0074] By repeatedly iterating the above-mentioned training, testing, and parameter adjustment processes, the system can select a set of hyperparameters with the best performance based on the loss function calculated each time. After determination, the model corresponding to this set of parameters is determined as the resource demand prediction model, so that both the sample data set and the training data set perform well in this model, thereby improving the prediction effect of the model.

[0075] In this embodiment, the system preferably uses a linear regression-based model to predict the resource requirements of the virtual machine, but the embodiment of the present invention does not limit the basic model of the prediction model, and the user can adjust it according to the needs.

[0076] Among them, the linear regression model can be expressed by the following mathematical formula:

[0077] Resource Demand=w1*CPU Ut ili zat i on+w2*Memory Usage+...+wn*Other Features+b

[0078] Among them, w1, w2, ..., wn are the weight parameters of the model, and b is the bias term.

[0079] In this embodiment, in order to train the model, the system divides the data set into a training set and a test set, and optimizes the parameters of the model through the training set to make it fit the training data better. Then, the generalization ability of the model is evaluated through the test set, that is, the prediction performance of the model on unseen data, so as to determine whether the model meets the requirements.

[0080] During the model training process, the system adjusts the parameters of the model by minimizing the loss function, where the loss function is used to measure the difference between the model prediction value and the true value. The loss functions selected in this embodiment are the root mean square error (Root Mean Square Error) and the mean absolute error (Mean Abso l uteError).

[0081] After completing the model training, the system also needs to evaluate and optimize the trained model by calculating the prediction error of the test set in the model to evaluate the model's prediction performance. If the prediction effect is poor, the model is tuned by adjusting the hyperparameters.

[0082] In this embodiment, the system mainly adjusts the model hyperparameters including learning rate, regularization parameter, number and size of hidden layers, number of iterations, batch size, etc. The purpose of adjusting the hyperparameters is to optimize the performance of the model, improve the generalization ability, and reduce the risk of overfitting or underfitting. Therefore, in the process of model evaluation and tuning, by adjusting the hyperparameters, you can try to improve the performance of the model so that it performs well on both the training set and the test set.

[0083] Determining the relationship between hyperparameters and prediction errors usually requires hyperparameter search, and common methods include grid search, random search, and Bayesian optimization. The above methods can be used to search for the best hyperparameter combination in the hyperparameter space to minimize the prediction error. The adjustment process for model hyperparameters is an iterative process that requires continuous trial of different hyperparameter combinations and evaluation of their performance on the validation set, ultimately selecting the best performing hyperparameter combination.

[0084] Specifically, the resource classification of the virtual machine according to the resource demand prediction in this embodiment specifically includes:

[0085] Generating a corresponding first resource allocation strategy according to the resource demand prediction, and allocating physical resources of the target virtual machine according to the first resource allocation strategy;

[0086] The energy consumption and the performance indicators of the target virtual machine are collected in real time, and the collected energy consumption and the performance indicators are input into the resource demand prediction model for prediction, and a corresponding second resource allocation strategy is generated according to the prediction data output by the model prediction, and the first resource allocation strategy is adjusted according to the second resource allocation strategy, and then the physical resources are allocated according to the adjusted strategy.

[0087] In order to achieve dynamic adjustment of resource allocation strategies, improve the degree of adaptation between resource allocation strategies and the actual situation of virtual machines, and reduce the energy consumption of virtual machines, the resource allocation method provided by the present invention will continuously collect energy consumption data and indicator data of virtual machines in real time after allocating resources to virtual machines according to the first allocation strategy, so that the prediction model can re-predict and output prediction data and generate a second resource allocation strategy, and then adjust the first strategy according to the newly generated second resource allocation strategy, so that the adjusted strategy is more in line with the current operating status and energy consumption of the virtual machine and the performance requirements of different virtual machines, thereby achieving dynamic optimization and adjustment of resource allocation strategies, further reducing the energy consumption and cost consumption of virtual machines, and building a green and low-carbon data center.

[0088] In order to better illustrate the working principle and step flow of a method and device for allocating resources to a virtual machine of the present invention, reference may be made to, but is not limited to, the relevant records above.

[0089] Accordingly, see Figure 2 , Figure 2 The structure diagram of an embodiment of a resource allocation device for a virtual machine provided by the present invention is shown in FIG. Figure 2 As shown, the resource allocation device includes a data collection and processing module 201 and a model training and prediction module 202.

[0090] Among them, the data acquisition and processing module 201 is used to acquire the real-time load, performance indicators and energy consumption of the target virtual machine in real time, and calculate the linear coefficient between each two data in the real-time load, the energy consumption and the performance indicators, output the corresponding three sub-data sets, and then merge the three sub-data sets and output them as the allocation data set.

[0091] Furthermore, the data acquisition and processing module 201 calculates the linear coefficient between each two data in the real-time load, the energy consumption and the performance index, outputs the corresponding three sub-data sets, and then merges the three sub-data sets and outputs them as a distribution data set, specifically including:

[0092] The real-time load, the energy consumption and the performance indicator are labeled respectively according to the collection time point, and the real-time load, the energy consumption and the performance indicator with the same collection time point after labeling are associated and combined; linear regression analysis is performed on the associated combined data according to the collection time point, and a corresponding number of linear coefficients are output, and then the linear coefficients of the real-time load and the energy consumption among the several linear coefficients are associated with the real-time load and the energy consumption as a first sub-dataset, the linear coefficients of the real-time load and the performance indicator among the several linear coefficients are associated with the real-time load and the performance indicator as a second sub-dataset, and the linear coefficients of the energy consumption and the performance indicator among the several linear coefficients are associated with the energy consumption and the performance indicator as a third sub-dataset.

[0093] The model training prediction module 202 is used to evenly divide the allocated data set into a sample data set and a training data set, input the training data set into a preset initial machine model for training, and after the training is completed, input the sample data set, target energy cost and target performance index into the trained resource demand prediction model, so that the resource demand prediction model outputs the resource demand prediction of the target virtual machine, and classifies the resources of the virtual machine according to the resource demand prediction.

[0094] Furthermore, the model training prediction module 202 inputs the training data set into a preset initial machine model for training, specifically:

[0095] The training data set is sequentially subjected to outlier removal, missing value supplementation and data normalization, and corresponding training sample data is output; feature extraction is performed on the training sample data and the attribute data of the target virtual machine, and corresponding feature subsets are output; and an importance analysis algorithm, a correlation analysis algorithm and a recursive feature elimination algorithm are called to sequentially perform screening processing on the feature subsets, and feature data is output;

[0096] The training sample data is evenly divided into a test set and a training set, the feature data and the training set are input into the initial machine model for training, and the model is tested with the test set after training, and the hyperparameters of the model are adjusted by the loss function until the data output by the model test satisfies.

[0097] Furthermore, the model training prediction module 202 inputs the feature data and the training set into the initial machine model for training, and tests the model through the test set after training, and adjusts the hyperparameters of the model through the loss function until the data output by the model test satisfies, specifically:

[0098] Input the feature data and the training set into the initial machine model for training, input the test set into the trained model to test the model, output first test data, and calculate the root mean square error and mean absolute error between the first test data and first real data as the loss function;

[0099] The learning rate, regularization parameter and number of iterations of the model are adjusted according to the calculated loss function, and the model iteration is trained, tested and hyperparameters adjusted until the loss function calculated and output by the test model is less than the preset loss function threshold. The model corresponding to the loss function is then output as a resource demand prediction model.

[0100] Furthermore, the model training prediction module 202 classifies the resources of the virtual machines according to the resource demand prediction, specifically including:

[0101] A corresponding first resource allocation strategy is generated according to the resource demand forecast, and the physical resources of the target virtual machine are allocated according to the first resource allocation strategy; the consumption data and indicator data of the target virtual machine are collected in real time, and the collected consumption data and indicator data are input into the resource demand forecasting model for forecasting, and a corresponding second resource allocation strategy is generated according to the forecast data output by the model forecasting, and the first resource allocation strategy is adjusted according to the second resource allocation strategy, and then the physical resources are allocated according to the adjusted strategy.

[0102] In summary, the embodiment of the present invention provides a resource allocation method and device for virtual machines, which generates three sub-data sets by calculating the linear coefficient between each two data in the real-time load, performance indicators and energy consumption collected from the target virtual machine, and integrates the three sub-data sets to output the allocation data set; the allocation data set is evenly divided into a sample data set and a training data set, the training data set is used to train the initial machine model, and the sample data set and the target energy cost and target performance indicators are input into the trained model for prediction, and resources are allocated according to the resource demand prediction output by the model. By calculating the linear coefficients between the three types of data, the correlation between the various parameters of the virtual machine is clarified, and the prediction accuracy and precision of the prediction model are improved; prediction and allocation based on real-time collected data realizes the dynamic adjustment of the virtual machine allocation strategy, and improves the practicality of resource allocation and the adequacy of resource utilization.

[0103] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A resource allocation method for a virtual machine, characterized in that: The following steps are involved: Acquire the real-time load, performance index and energy consumption of the target virtual machine in real time, calculate the linear coefficient between each two data of the real-time load, the energy consumption and the performance index, output the corresponding three sub-data sets, and then merge the three sub-data sets and output them as the allocation data set; The allocated data set is evenly divided into a sample data set and a training data set, the training data set is input into a preset initial machine model for training, and after the training is completed, the sample data set, the target energy cost and the target performance index are input into the trained resource demand prediction model for prediction and output of the resource demand prediction of the target virtual machine, and then the virtual machine is classified according to the resource demand prediction.

2. A method for allocating resources to a virtual machine as claimed in claim 1, characterized in that: The calculating of the linear coefficient between each two data of the real-time load, the energy consumption and the performance index, outputting three corresponding sub-data sets, and then merging the three sub-data sets and outputting them as a distribution data set specifically includes: Labeling the real-time load, the energy consumption and the performance index according to the collection time points respectively, and associating and combining the real-time load, the energy consumption and the performance index having the same collection time points after labeling; Perform linear regression analysis on the associated combined data according to the collection time points, output corresponding linear coefficients, and then associate the linear coefficients about the real-time load and the energy consumption among the linear coefficients with the real-time load and the energy consumption as a first sub-dataset, associate the linear coefficients about the real-time load and the performance index among the linear coefficients with the real-time load and the performance index as a second sub-dataset, and associate the linear coefficients about the energy consumption and the performance index among the linear coefficients with the energy consumption and the performance index as a third sub-dataset.

3. A method for allocating resources to a virtual machine as claimed in claim 1, characterized in that: The step of inputting the training data set into a preset initial machine model for training is specifically as follows: Performing outlier removal, missing value supplementation and data normalization on the training data set in turn, and outputting corresponding training sample data; Extracting features from the training sample data and the attribute data of the target virtual machine respectively, outputting corresponding feature subsets, and calling an importance analysis algorithm, a correlation analysis algorithm, and a recursive feature elimination algorithm to sequentially screen and process the feature subsets, and outputting feature data; The training sample data is evenly divided into a test set and a training set, the feature data and the training set are input into the initial machine model for training, and the model is tested with the test set after training, and the hyperparameters of the model are adjusted by the loss function until the data output by the model test satisfies.

4. A method for allocating resources to a virtual machine as claimed in claim 3, characterized in that: The feature data and the training set are input into the initial machine model for training, and the model is tested by the test set after training, and the hyperparameters of the model are adjusted by the loss function until the data output by the model test satisfies the requirements, specifically: Input the feature data and the training set into the initial machine model for training, input the test set into the trained model to test the model, output first test data, and calculate the root mean square error and mean absolute error between the first test data and first real data as the loss function; The learning rate, regularization parameter and number of iterations of the model are adjusted according to the calculated loss function, and the model iteration is trained, tested and hyperparameters adjusted until the loss function calculated and output by the test model is less than the preset loss function threshold. The model corresponding to the loss function is then output as a resource demand prediction model.

5. A method for allocating resources to a virtual machine as claimed in claim 1, characterized in that: The classifying the resources of the virtual machines according to the resource demand prediction specifically includes: Generating a corresponding first resource allocation strategy according to the resource demand prediction, and allocating physical resources of the target virtual machine according to the first resource allocation strategy; The energy consumption and the performance indicators of the target virtual machine are collected in real time, and the collected energy consumption and the performance indicators are input into the resource demand prediction model for prediction, and a corresponding second resource allocation strategy is generated according to the prediction data output by the model prediction, and the first resource allocation strategy is adjusted according to the second resource allocation strategy, and then the physical resources are allocated according to the adjusted strategy.

6. A resource allocation device for a virtual machine, characterized in that: The resource allocation device includes a data acquisition and processing module and a model training and prediction module; The data acquisition and processing module is used to acquire the real-time load, performance index and energy consumption of the target virtual machine in real time, calculate the linear coefficient between each two data in the real-time load, the energy consumption and the performance index, output the corresponding three sub-data sets, and then merge the three sub-data sets and output them as the allocation data set; The model training prediction module is used to evenly divide the allocated data set into a sample data set and a training data set, input the training data set into a preset initial machine model for training, and after the training is completed, input the sample data set, target energy cost and target performance index into the trained resource demand prediction model, so that the resource demand prediction model outputs the resource demand prediction of the target virtual machine, and classifies the resources of the virtual machine according to the resource demand prediction.

7. A resource allocation device for a virtual machine as claimed in claim 6, characterized in that: The data acquisition and processing module calculates the linear coefficient between each two data in the real-time load, the energy consumption and the performance index, outputs three corresponding sub-data sets, and then merges the three sub-data sets and outputs them as a distribution data set, specifically including: Labeling the real-time load, the energy consumption and the performance index according to the collection time points respectively, and associating and combining the real-time load, the energy consumption and the performance index having the same collection time points after labeling; Perform linear regression analysis on the associated combined data according to the collection time points, output corresponding linear coefficients, and then associate the linear coefficients about the real-time load and the energy consumption among the linear coefficients with the real-time load and the energy consumption as a first sub-dataset, associate the linear coefficients about the real-time load and the performance index among the linear coefficients with the real-time load and the performance index as a second sub-dataset, and associate the linear coefficients about the energy consumption and the performance index among the linear coefficients with the energy consumption and the performance index as a third sub-dataset.

8. A resource allocation device for a virtual machine as claimed in claim 6, characterized in that: The model training prediction module inputs the training data set into a preset initial machine model for training, specifically: Performing outlier removal, missing value supplementation and data normalization on the training data set in turn, and outputting corresponding training sample data; Extracting features from the training sample data and the attribute data of the target virtual machine respectively, outputting corresponding feature subsets, and calling an importance analysis algorithm, a correlation analysis algorithm, and a recursive feature elimination algorithm to sequentially screen and process the feature subsets, and outputting feature data; The training sample data is evenly divided into a test set and a training set, the feature data and the training set are input into the initial machine model for training, and the model is tested with the test set after training, and the hyperparameters of the model are adjusted by the loss function until the data output by the model test satisfies.

9. A resource allocation device for a virtual machine as claimed in claim 8, characterized in that: The model training prediction module inputs the feature data and the training set into the initial machine model for training, and tests the model through the test set after training, and adjusts the hyperparameters of the model through the loss function until the data output by the model test satisfies, specifically: Input the feature data and the training set into the initial machine model for training, input the test set into the trained model to test the model, output first test data, and calculate the root mean square error and mean absolute error between the first test data and first real data as the loss function; The learning rate, regularization parameter and number of iterations of the model are adjusted according to the calculated loss function, and the model iteration is trained, tested and hyperparameters adjusted until the loss function calculated and output by the test model is less than the preset loss function threshold. The model corresponding to the loss function is then output as a resource demand prediction model.

10. The resource allocation device for a virtual machine according to claim 6, characterized in that: The model training prediction module classifies the resources of the virtual machine according to the resource demand prediction, specifically including: Generating a corresponding first resource allocation strategy according to the resource demand prediction, and allocating physical resources of the target virtual machine according to the first resource allocation strategy; The consumption data and indicator data of the target virtual machine are collected in real time, and the collected consumption data and indicator data are input into the resource demand prediction model for prediction, and a corresponding second resource allocation strategy is generated according to the prediction data output by the model prediction, and the first resource allocation strategy is adjusted according to the second resource allocation strategy, and then the physical resources are allocated according to the adjusted strategy.

Images