Cloud server aging prediction method based on self-attention mechanism DLSTM

Abstract

The present invention discloses a cloud server aging prediction method based on a self-attention mechanism DLSTM, comprising the following steps: step 1, collecting data indicators of cloud server aging conditions, and obtaining time series data of cloud server resources and performance parameters; step 2, preprocessing the sequence data to obtain a preprocessed data set; step 3, dividing the cloud server aging data preprocessed in step 2 into a training set and a test set; step 4, constructing a DLSTM prediction model of cloud server aging data time series based on an attention mechanism; step 5, training the DLSTM prediction model using the training set data; step 6, predicting the test set data using the trained DLSTM prediction model, and evaluating the performance of the DLSTM prediction model. The present invention solves the problem that traditional prediction methods are not accurate enough in predicting the aging conditions of cloud servers that run for a long time and have a large amount of data.

Description

Technical Field

[0001] The present invention belongs to the technical field of time series prediction and relates to a cloud server aging prediction method based on a self-attention mechanism DLSTM (Deep Long Short-Term Memory). Background Art

[0002] Cloud computing includes a variety of computing resources, providing secure and fast cloud computing services and data storage services. Cloud server is one of the important supporting technologies for cloud computing. Its high scalability, high flexibility and high cost-effectiveness enable people to obtain corresponding services according to their needs, which can save costs and improve resource utilization. However, in the continuous operation of the server, software aging begins to appear. Software aging is caused by the accumulation of errors such as resource leakage, unreleased file locks and unterminated processes, which reduces system performance or even crashes the system, and ultimately leads to a sharp increase in system overhead. In order to further optimize the use of cloud servers, it is very important to explore the impact of cloud server aging on cloud server performance.

[0003] With the continuous development of cloud computing technology and cloud servers, more and more experts have begun to focus on studying the laws of cloud server aging and exploring how to make cloud servers provide services to cloud users in a better state.

[0004] There are two types of existing aging prediction and analysis methods: state model-based and data measurement-based. Among them, the state model-based method mainly uses Petri nets and Markov modeling methods to establish a system state model, but in actual tasks, the process of establishing an accurate model is very difficult; the data measurement-based method mainly includes time series methods and machine learning methods. Among them, the long short-term memory (LSTM) deep learning network can meet the needs of memorizing data information for a long period of time to achieve prediction effects, but it does not perform well in time series prediction. As for the traditional BP neural network method, its parameter selection is more difficult, and it is difficult to select the most appropriate parameters to achieve the best prediction effect. Summary of the invention

[0005] The purpose of the present invention is to provide a cloud server aging prediction method based on a self-attention mechanism DLSTM, which solves the problem that traditional prediction methods are not accurate enough in predicting the aging of cloud servers that run for a long time and have a large amount of data.

[0006] The technical solution adopted by the present invention is:

[0007] The cloud server aging prediction method based on the self-attention mechanism DLSTM includes the following steps:

[0008] Step 1: Collect data indicators of cloud server aging and obtain time series data of cloud server resources and performance parameters;

[0009] Step 2, preprocessing the sequence data to obtain a preprocessed data set;

[0010] Step 3, dividing the cloud server aging data preprocessed in step 2 into a training set and a test set;

[0011] Step 4: Build a DLSTM prediction model for cloud server aging data time series based on the attention mechanism;

[0012] Step 5, using the training set data to train the DLSTM prediction model;

[0013] Step 6: Use the trained DLSTM prediction model to predict the test set data and evaluate the performance of the DLSTM prediction model.

[0014] The present invention is also characterized in that:

[0015] Step 2 specifically includes:

[0016] Step 2.1, perform first-order difference on the sequence data to obtain the difference sequence;

[0017] Step 2.2, convert the first-order difference data sequence into a time step matrix, where each element in the matrix contains a data segment of the length of a time step;

[0018] Step 2.3, normalize the time step matrix to the interval [-1,1] to obtain the preprocessed data set.

[0019] The specific process of step 2.2 is:

[0020] Convert the original sequence into an n*1 matrix P1; insert a 0 before the original sequence and convert it into an n*1 matrix P2; merge the matrices P1 and P2 into an n*2 matrix P′; that is:

[0021]

[0022] The matrix P′ is the time step matrix.

[0023] In step 4, the DLSTM prediction model is composed of 50 stacked LSTMs. Each LSTM includes a forget gate, an input gate, and an output gate. The activation function of the input gate is the tanh function, and the activation functions used for the forget gate and the output gate are the sigmoid function. The DLSTM neural network includes an input layer, a hidden layer, a connection activation layer, and an output layer connected in sequence, and a dropout layer is set. The input layer is encapsulated with an attention mechanism.

[0024] The encapsulation process of the attention mechanism is as follows: first, the exchange layer is used to convert the format of the input data into the expected format; then the Softmax activation function in the dense layer is used to calculate the weight of the feature, where the Lambda layer uses tf.keras.backend.mean() to calculate the average value of the tensor; then the exchange layer is used to convert the output of the dense layer into the format required by the product layer; finally, the product layer is reached to multiply the weight by the input to complete the encapsulation of the attention mechanism.

[0025] The beneficial effects of the present invention are:

[0026] The present invention can make better use of the important features of the time series and solve the problem that the traditional prediction method has low prediction accuracy for data with large fluctuations. It is proposed to add a self-attention mechanism to the input layer, which can give different weights to the input features in the time series. When predicting, multiple features with higher weights are used as the input of a feature prediction, thereby improving the prediction accuracy. The attention mechanism can adaptively select the data sequence related to the corresponding time point according to the attention weight calculation and weight distribution of the dense layer on the input time series, so that the model can select more important input features by itself and obtain the long time series characteristics of the time series; the DLSTM method is proposed. Compared with the traditional LSTM method, each layer of LSTM in DLSTM runs on different time scales and transmits the results to the next layer of LSTM, so that DLSTM can effectively utilize the characteristics of each layer of LSTM, extract information on the time series from different scales, and thus learn more complex time series data. Therefore, when predicting a large amount of data, the prediction accuracy of DLSTM will be higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is an overall framework diagram of the cloud server aging prediction method based on the self-attention mechanism DLSTM of the present invention;

[0028] Figure 2 It is a DLSTM structure diagram of the cloud server aging prediction method based on the self-attention mechanism DLSTM of the present invention;

[0029] Figure 3 The attention mechanism encapsulation flow chart of the cloud server aging prediction method based on the self-attention mechanism DLSTM of the present invention;

[0030] Figure 4 It is a time series diagram of raw data in an embodiment of the present invention;

[0031] Figure 5 This is a diagram of prediction results according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0033] like Figure 1 to Figure 3 The present invention provides a cloud server aging prediction method based on a self-attention mechanism DLSTM, comprising the following steps:

[0034] Step 1: Collect data indicators of cloud server aging and obtain time series data of cloud server resources and performance parameters;

[0035] Step 2, preprocessing the sequence data to obtain a preprocessed data set;

[0036] Step 3, dividing the cloud server aging data preprocessed in step 2 into a training set and a test set;

[0037] Step 4: Build a DLSTM prediction model for cloud server aging data time series based on the attention mechanism;

[0038] Step 5, using the training set data to train the DLSTM prediction model;

[0039] Step 6: Use the trained DLSTM prediction model to predict the test set data and evaluate the performance of the DLSTM prediction model.

[0040] The time series data of cloud server performance in step 1 is free memory;

[0041] The specific preprocessing process in step 2 is:

[0042] Step 2.1, perform first-order difference on the sequence data to obtain the difference sequence;

[0043] First, perform first-order difference on the original data; let the time series of the original cloud server performance parameters be X = (x1, x2, ..., x n )(n is the length of the entire time series), the data sequence after difference is Y=(y1,y2,…,y n-1 ); use the later value in the sequence minus the previous value, that is:

[0044] y i =x i+1 -x i (1)

[0045] Formula (1) is used to obtain the first-order difference data series Y, thereby eliminating the dependence of the time series on time.

[0046] Step 2.2, convert the first-order difference data sequence into a time step matrix, where each element in the matrix contains a data segment of the length of a time step;

[0047] The time step matrix is ​​used for prediction; the time step used in this scheme is 2, and the construction process is: convert the original sequence into an n*1 matrix P1; insert a 0 before the original sequence, and then convert it into an n*1 matrix P2; merge the matrices P1 and P2 into an n*2 matrix P′; that is:

[0048]

[0049] Step 2.3, normalize the time step matrix to the interval [-1,1] to obtain the preprocessed data set, specifically:

[0050] use Represents x i The normalized value, |x| max The maximum absolute value of the data after difference is obtained, and the data in the matrix P′ is normalized to the interval [-1,1].

[0051] The construction of the DLSTM prediction model in step 4 is as follows:

[0052] The DLSTM prediction model is composed of 50 stacked LSTMs. Each LSTM includes a forget gate, an input gate, and an output gate. The activation function of the input gate is the tanh function, and the activation functions used for the forget gate and the output gate are the sigmoid function. The DLSTM neural network includes an input layer, a hidden layer, a connection activation layer, and an output layer that are connected in sequence, and a dropout layer is set.

[0053] Each LSTM maintains the traditional structure; let the input of the DLSTM model input layer at time t be x t , the output of the output layer is h t ; To prevent the model from overfitting, a dropout layer is set so that during forward propagation, the activation value of the neuron stops working with a certain probability p. In this scheme, p is set to 0.3;

[0054] The activation layer is connected after the hidden layer to make the matrix operation result nonlinear; the activation function used by the forget gate and output gate in LSTM is the Sigmoid function, that is, This function outputs a value of 0 or 1, where output 0 means discarding the current information and output 1 means retaining the current information;

[0055] The input gate activation function is the tanh function, that is, Used to calculate candidate value vector information;

[0056] The input of the i-th LSTM layer of the DLSTM prediction model at time t is x t , output h at time t-1 t-1 、Module state C at time t-1 t-1and i-1 layer LSTM weight information W (i-1) Together they form; output h at time t t and state C t Passed to time t+1; weight information W of the i-th layer LSTM at time t (i) Passed to the next layer of LSTM to assist prediction until the last layer of LSTM obtains the output value; at time t, the weight information W of the first layer of LSTM (1) The second layer LSTM is passed in as input, and so on, until the last LSTM outputs the result;

[0057] Among them, LSTM has a forget gate, an input gate, and an output gate; the calculation method of the forget gate is:

[0058] f t =σ(W f (i-1) ·[h t-1 ,x t ]+b f ) (3)

[0059] The input gate calculation method is:

[0060] i t =σ(W i (i-1) ·[h t-1 ,x t ]+b i ) (4)

[0061] The candidate value vector is calculated as follows:

[0062]

[0063] The status output is:

[0064]

[0065] Output gate calculation method:

[0066] o t =σ(W o (i-1) ·[h t-1 ,x t ]+b o ) (7)

[0067] The output is:

[0068] h t =o t *tanh(C t ) (8);

[0069] In formulas (4) to (9), W is the weight information, ht is the output at time t, x t is the input at time t, W (i-1) is the weight information of the i-1th layer LSTM, and b is the bias term.

[0070] In step 4, the specific implementation method of adding the attention mechanism to the input layer of the DLSTM prediction model of the cloud server aging data time series is used to calculate the weighted average of the input information using a dense layer with a Softmax activation function to complete the allocation of attention weights. The encapsulation of the attention mechanism involves the comprehensive use of multiple layers: the exchange layer transposes the dimension of the input value into the desired dimension; the Lambda layer uses a custom function to implement the required specific functions to achieve the desired effect; the RepeatVector layer repeats the input data n times.

[0071] The encapsulation process of the attention mechanism is as follows: first, the exchange layer is used to convert the format of the input data into the expected format; then the Softmax activation function in the dense layer is used to calculate the weight of the feature, where the Lambda layer uses tf.keras.backend.mean() to calculate the average value of the tensor; then the exchange layer is used to convert the output of the dense layer into the format required by the product layer; finally, the product layer is reached to multiply the weight with the input to complete the encapsulation of the attention mechanism;

[0072] In step 6, the root mean square error RMSE, mean absolute percentage error MAPE and mean absolute error MAE are used as evaluation indicators, and the formulas are shown in (9)-(11) respectively;

[0073]

[0074]

[0075]

[0076] Where N is the length of the data, y i is the predicted value, x i This is the raw data that is aged by the cloud server.

[0077] Example

[0078] This example uses free memory as the aging indicator and collects the time series data of free memory of the actual running cloud server. The values ​​are plotted every 20 points, such as Figure 4 The prediction results of the cloud server aging prediction method based on the self-attention mechanism DLSTM are compared with the original data of the cloud server. Figure 5 The specific steps are as follows:

[0079] Step 1, collecting data indicators of cloud server aging, obtaining time series data of cloud server resources and performance parameters, wherein the resources and performance parameters are: free memory;

[0080] Step 2: Preprocess the sequence data.

[0081] Step 2.1, perform first-order difference on the sequence data to obtain the difference sequence;

[0082] Step 2.2, convert the first-order difference data sequence into a time step matrix, where each element contains a data segment of the length of a time step. The time step used in this scheme is 2;

[0083] Step 2.3, normalize the time step matrix to [-1, 1] to complete the preprocessing.

[0084] Step 3, dividing the cloud server aging data preprocessed in step 2 into a training set and a test set;

[0085] Step 4: Build a DLSTM prediction model for the cloud server aging data time series based on the attention mechanism.

[0086] Step 4.1, build a DLSTM prediction model;

[0087] Step 4.2, use the exchange layer to convert the format of the input data into the expected format;

[0088] Step 4.3, then use the Softmax activation function in the dense layer to calculate the weight of the feature, where the Lambda layer uses tf.keras.backend.mean() to calculate the average value of the tensor;

[0089] Step 4.4, use the exchange layer to convert the output of the dense layer into the format required by the product layer;

[0090] Step 4.5, multiply the weight and input in the product layer to complete the encapsulation of the attention mechanism;

[0091] Step 5, using the training set data to train the DLSTM prediction model;

[0092] Step 6: Use the trained DLSTM prediction model to predict the test set data and evaluate the performance of the DLSTM prediction model.

[0093] The root mean square error RMSE, mean absolute percentage error MAPE and mean absolute error MAE are used as evaluation indicators, and the formulas are shown in (9)-(11) respectively;

[0094]

[0095]

[0096]

[0097] Where N is the length of the data, y i is the predicted value, x i is the original data of cloud server aging. The error results are shown in Table 1.

[0098] Table 1 Model prediction error values

[0099] Prediction Model RMSE MAE MAPE Attention-based DLSTM model 4770.41 621.05 0.08

Claims

1. A cloud server aging prediction method based on self-attention mechanism DLSTM, characterized by: The steps include: Step 1: Collect data indicators of cloud server aging and obtain time series data of cloud server resources and performance parameters; Step 2, preprocessing the sequence data to obtain a preprocessed data set; Step 3, dividing the cloud server aging data preprocessed in step 2 into a training set and a test set; Step 4: Build a DLSTM prediction model for cloud server aging data time series based on the attention mechanism; Step 5, using the training set data to train the DLSTM prediction model; Step 6: Use the trained DLSTM prediction model to predict the test set data and evaluate the performance of the DLSTM prediction model; The preprocessing process in step 2 is specifically as follows: Step 2.1, perform first-order difference on the sequence data to obtain the difference sequence; First, perform first-order difference on the original data; let the time series of the original cloud server performance parameters be X = (x1, x2, ..., x n ), n is the length of the entire time series, and the data sequence after difference is Y=(y1,y2,…,y n-1 ); use the later value in the sequence minus the previous value, that is: y i =x i+1 -x i (1) Formula (1) is used to obtain the first-order difference data series Y, thereby eliminating the dependence of the time series on time; Step 2.2, convert the first-order difference data sequence into a time step matrix, where each element in the matrix contains a data segment of the length of a time step; Step 2.3, normalize the time step matrix to the interval [-1,1] to obtain the preprocessed data set, specifically: use Represents x i The normalized value, |x| max is the maximum value of the absolute value of the data after difference, and the data in the matrix P′ is normalized to the interval [-1,1]; The specific process of step 2.2 is as follows: Convert the original sequence into an n*1 matrix P1; insert a 0 before the original sequence and convert it into an n*1 matrix P2; merge the matrices P1 and P2 into an n*2 matrix P′; that is: The matrix P′ is the time step matrix; The DLSTM prediction model in step 4 is composed of 50 LSTM stacks, each of which includes a forget gate, an input gate and an output gate, the activation function of the input gate is a tanh function, and the activation functions used by the forget gate and the output gate are sigmoid functions; the DLSTM neural network includes an input layer, a hidden layer, a connection activation layer, and an output layer connected in sequence, and is provided with a dropout layer; the input layer is encapsulated with an attention mechanism; The encapsulation process of the attention mechanism is as follows: first, the exchange layer is used to convert the format of the input data into the expected format; then the Softmax activation function in the dense layer is used to calculate the weight of the feature, where the Lambda layer uses tf.keras.backend.mean() to calculate the average value of the tensor; then the exchange layer is used to convert the output of the dense layer into the format required by the product layer; finally, the product layer is reached to multiply the weight by the input to complete the encapsulation of the attention mechanism.

2. The cloud server aging prediction method based on the self-attention mechanism DLSTM as claimed in claim 1 is characterized in that: The step 2 specifically includes: Step 2.1, perform first-order difference on the sequence data to obtain the difference sequence; Step 2.2, convert the first-order difference data sequence into a time step matrix, where each element in the matrix contains a data segment of the length of a time step; Step 2.3, normalize the time step matrix to the interval [-1,1] to obtain the preprocessed data set.

Images