Method for training load prediction model, load prediction method and device of server

Abstract

The application discloses a training method of a load prediction model, a load prediction method and device of a server, the model comprising an encoder and a decoder, the encoder comprising a first graph convolutional neural network, a first feedforward neural network, a first multi-head attention mechanism network and a second feedforward neural network, and the decoder comprising a second graph convolutional neural network, a third feedforward neural network, a second multi-head attention mechanism network and a fourth feedforward neural network. The method comprises sequentially inputting each graph structure in each training sample into each network of the encoder to obtain a first network space-time feature after convolution calculation of each training sample, sequentially inputting each graph structure in each true value sample into the second graph convolutional neural network and the third feedforward neural network to obtain a second network space feature after convolution calculation, and sequentially inputting the first network space-time feature and the second network space feature into the second multi-head attention mechanism network and the fourth feedforward neural network, and finally obtaining the model through adjustment training.

Description

Technical Field

[0001] This application relates to the field of cloud computing technology, and more specifically, to a method for training a load prediction model, a method for predicting server load, and an apparatus. Background Technology

[0002] Cloud computing platforms represent a significant development trend in the information technology field. By providing elastic resources and flexible service models, they meet the needs of businesses and individuals for efficient, scalable, and reliable computing capabilities. Server load forecasting is a critical task within cloud computing platforms. It helps cloud service providers effectively plan and manage resources, ensuring service quality for users while simultaneously reducing costs and increasing profits.

[0003] In related technologies, many load forecasting models based on statistical methods have been proposed and applied, such as time series models (e.g., ARIMA (Autoregressive Integrated Moving Average) and SARIMA (Seasonal Autoregressive Integrated Moving Average)), regression models (e.g., linear regression, support vector machines), and some deep learning models (e.g., recurrent neural networks, long short-term memory networks, and convolutional neural networks). However, these traditional methods often struggle to handle complex nonlinear relationships and spatiotemporal dependencies, thus limiting their predictive accuracy. Summary of the Invention

[0004] This application provides a method for training a load prediction model, a method and apparatus for predicting server load, which can improve the accuracy of load prediction.

[0005] The specific technical solution is as follows:

[0006] In a first aspect, embodiments of this application provide a method for training a load prediction model. The load prediction model includes an encoder and a decoder. The encoder includes a first graph convolutional neural network, a first feedforward neural network, a first multi-head attention mechanism network, and a second feedforward neural network. The decoder includes a second graph convolutional neural network, a third feedforward neural network, a second multi-head attention mechanism network, and a fourth feedforward neural network. The method includes:

[0007] Obtain a training set, wherein the training set includes multiple training samples and ground truth samples corresponding to each training sample, each training sample includes a graph structure of multiple consecutive historical moments, the ground truth samples corresponding to each training sample include a graph structure of multiple future moments adjacent to the multiple consecutive historical moments, and the graph structure of each moment includes target data information of each of the multiple servers at that moment, the target data information including at least load information;

[0008] Each graph structure in each training sample is input into the first graph convolutional neural network to extract the first road network spatial features of each graph structure in each training sample; and each graph structure in each ground value sample is input into the second graph convolutional neural network to extract the second road network spatial features of each graph structure in each ground value sample.

[0009] Each of the first road network spatial features is processed by the first feedforward neural network to obtain the first road network spatial features after convolution calculation, and each of the second road network spatial features is processed by the third feedforward neural network to obtain the second road network spatial features after convolution calculation.

[0010] Add corresponding temporal information to the first network spatial features calculated by convolution at each time step, and input the first network spatial features of each training sample with added temporal information into the first multi-head attention mechanism network to obtain the first network spatiotemporal features of each training sample;

[0011] The first network spatiotemporal features of each training sample are processed by convolution of the second feedforward neural network to obtain the first network spatiotemporal features of each training sample after convolution.

[0012] The first network spatiotemporal features calculated by convolution, and multiple second network spatial features in the corresponding ground truth samples after adding time information, are input into the second multi-head attention mechanism network to obtain the third network spatiotemporal features of multiple future time moments corresponding to each training sample, and the second network spatiotemporal features of each ground truth sample.

[0013] The third network spatiotemporal features of each training sample at multiple future time points, and the second network spatiotemporal features of each ground value sample are input into the fourth feedforward neural network to obtain the third network spatiotemporal features and the second network spatiotemporal features after convolution calculation.

[0014] Based on the spatiotemporal features of the third network after convolution and the spatiotemporal features of the second network after convolution, the current loss value is calculated. If the current loss value does not meet the convergence condition, the model parameters of the load prediction model are adjusted, and the load prediction model with adjusted parameters is trained until the current loss value meets the convergence condition, thus obtaining the final load prediction model.

[0015] In one possible implementation, the method for obtaining target data information for each of the servers includes:

[0016] Anomaly detection is performed on the raw data information of each of the servers;

[0017] When an outlier is detected in the original data, the outlier is repaired.

[0018] The original data information after outlier repair is normalized or standardized to obtain the target data information.

[0019] In one possible implementation, the outlier detection of the raw data information of each of the servers includes:

[0020] The original data information is dimensionality reduced based on the principal component analysis (PCA) algorithm, and outlier detection is performed on the dimensionality-reduced original data information using the local outlier factor (LOF).

[0021] In one possible implementation, the repair of the outlier includes:

[0022] Generative adversarial networks are used to repair the outliers to obtain the first data after repair, and support vector regression algorithms are used to repair the outliers to obtain the second data after repair.

[0023] The first data and the second data are weighted and calculated to obtain the final result after the outlier is repaired.

[0024] In one possible implementation, the target data information further includes: traffic and / or network performance.

[0025] Secondly, embodiments of this application provide a server load prediction method, the method comprising:

[0026] Obtain target data information of each server in the cloud computing platform at multiple recent historical moments, wherein the target data information includes at least load information;

[0027] For each historical moment, a graph structure is generated based on the target data information of each server in the cloud computing platform;

[0028] The load prediction model is input into the graph structure of the most recent historical moments to predict the load information of each server at multiple future moments, wherein the load prediction model is trained according to the method described in any embodiment of the first aspect.

[0029] Thirdly, embodiments of this application provide a training apparatus for a load prediction model. The load prediction model includes an encoder and a decoder. The encoder includes a first graph convolutional neural network, a first feedforward neural network, a first multi-head attention mechanism network, and a second feedforward neural network. The decoder includes a second graph convolutional neural network, a third feedforward neural network, a second multi-head attention mechanism network, and a fourth feedforward neural network. The apparatus includes:

[0030] An acquisition unit is used to acquire a training set, wherein the training set includes multiple training samples and ground value samples corresponding to each training sample, each training sample includes a graph structure of multiple consecutive historical moments, the ground value samples corresponding to each training sample include a graph structure of multiple future moments adjacent to the multiple consecutive historical moments, and the graph structure of each moment includes target data information of each of the multiple servers at that moment, the target data information including at least load information.

[0031] The first feature extraction unit is used to input each graph structure in each training sample into the first graph convolutional neural network to extract the first road network spatial features of each graph structure in each training sample.

[0032] The second feature extraction unit is used to input each graph structure in each ground value sample into the second graph convolutional neural network to extract the second road network spatial features of each graph structure in each ground value sample;

[0033] The first convolutional computation unit is used to perform convolutional computation on each of the first road network spatial features through the first feedforward neural network to obtain the first road network spatial features after convolutional computation.

[0034] The second convolutional computation unit is used to perform convolutional computation on each of the second network spatial features through the third feedforward neural network to obtain the second network spatial features after convolutional computation.

[0035] Add units to add corresponding temporal information to the first network spatial features after convolution calculation at each time step;

[0036] The first attention processing unit is used to input the first road network spatial features of each training sample after adding time information into the first multi-head attention mechanism network to obtain the first road network spatiotemporal features of each training sample.

[0037] The third convolutional computation unit is used to perform convolutional computation on the first network spatiotemporal features of each training sample through the second feedforward neural network to obtain the first network spatiotemporal features of each training sample after convolutional computation.

[0038] The second attention processing unit is used to input the first network spatiotemporal features after convolution calculation and multiple second network spatial features in the corresponding ground value samples after adding time information into the second multi-head attention mechanism network to obtain the third network spatiotemporal features of multiple future times corresponding to each training sample, and the second network spatiotemporal features of each ground value sample.

[0039] The fourth convolutional computation unit is used to input the third network spatiotemporal features of multiple future time points corresponding to each training sample, and the second network spatiotemporal features of each ground value sample into the fourth feedforward neural network to obtain the third network spatiotemporal features and the second network spatiotemporal features after convolutional computation.

[0040] The training unit is adjusted to calculate the current loss value based on the spatiotemporal features of the third network after convolution and the spatiotemporal features of the second network after convolution. When the current loss value does not meet the convergence condition, the model parameters of the load prediction model are adjusted, and the load prediction model with adjusted parameters is trained until the current loss value meets the convergence condition, thus obtaining the final load prediction model.

[0041] In one possible implementation, the acquisition unit includes:

[0042] An outlier detection module is used to detect outliers in the raw data information of each server.

[0043] The repair module is used to repair the abnormal values ​​when an abnormal value is detected in the original data information.

[0044] The processing module is used to normalize or standardize the original data information after outlier repair to obtain the target data information.

[0045] In one possible implementation, the outlier detection module is used to perform dimensionality reduction processing on the original data information based on the principal component analysis (PCA) algorithm, and to perform outlier detection on the dimensionality-reduced original data information using the local outlier factor (LOF).

[0046] In one possible implementation, the repair module is used to repair the outliers using a generative adversarial network to obtain repaired first data, and to repair the outliers using a support vector regression algorithm to obtain repaired second data; the first data and the second data are weighted and calculated to obtain the final result after the outlier repair.

[0047] In one possible implementation, the target data information further includes: traffic and / or network performance.

[0048] Fourthly, embodiments of this application provide a server load prediction device, the device comprising:

[0049] The acquisition unit is used to acquire target data information of each server in the cloud computing platform at multiple recent historical moments, wherein the target data information includes at least load information.

[0050] A generation unit is used to generate a graph structure for each historical moment based on the target data information of each server in the cloud computing platform;

[0051] The prediction unit is used to input the graph structure of multiple recent historical moments into the load prediction model to predict the load information of each server at multiple future moments, wherein the load prediction model is trained according to the method described in any embodiment of the first aspect.

[0052] Fifthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any possible implementation of the first aspect, and implements the method as described in any possible implementation of the second aspect.

[0053] Sixthly, embodiments of this application provide an electronic device, which includes:

[0054] One or more processors;

[0055] The processor is coupled to a storage device for storing one or more programs;

[0056] When one or more programs are executed by one or more processors, the electronic device implements the method as in any possible implementation of the first aspect, implements the method as in any possible implementation of the first aspect, and implements the method as in any possible implementation of the second aspect.

[0057] In a seventh aspect, embodiments of this application provide a computer program product containing instructions that, when executed on a computer or processor, cause the computer or processor to perform the method described in any possible implementation of the first aspect, thereby implementing the method described in any possible implementation of the second aspect.

[0058] The load prediction model training method, server load prediction method, and apparatus provided in this application include an encoder and a decoder. Both the encoder and decoder incorporate a graph convolutional neural network, a feedforward neural network, and a multi-head attention mechanism network. In each training sample, each graph structure sequentially passes through the first graph convolutional neural network, the first feedforward neural network, the first multi-head attention mechanism network, and the second feedforward neural network in the encoder. The final output is the first network spatiotemporal feature after convolutional calculation for each training sample. In each ground value sample, each graph structure is sequentially input into the second graph convolutional neural network and the third feedforward neural network to obtain the second network spatial feature after convolutional calculation. The third-path network spatiotemporal features at multiple future time points corresponding to each training sample, and the second-path network spatiotemporal features of each ground truth sample are input into a fourth feedforward neural network to obtain the convolutionally computed third-path network spatiotemporal features and the convolutionally computed second-path network spatiotemporal features. Based on these features, the current loss value is calculated. If the current loss value does not meet the convergence condition, the model parameters of the load prediction model are adjusted, and the adjusted load prediction model is trained again until the current loss value meets the convergence condition, thus obtaining the final load prediction model for load prediction. Therefore, this embodiment can analyze the load information of each server in a cloud computing platform from both spatial and temporal perspectives, capturing long-term dependencies, thereby enabling the trained load prediction model to more accurately predict server load information.

[0059] The innovativeness of this application's embodiments includes at least the following:

[0060] 1. The load prediction model provided in this application includes an encoder and a decoder, both of which incorporate Graph Convolutional Networks (GCNs), feedforward neural networks, and multi-head attention mechanisms. This application first transforms the target data information into a graph structure including the server topology. Then, it extracts road network spatial features from the spatial dimension based on the GCN, and further analyzes the dependencies between various road network spatial features from the temporal dimension based on the multi-head attention mechanism, extracting spatiotemporal features of the road network. After each feature extraction, the feedforward neural network refines the extracted features, accelerating the convergence of the load prediction model and improving its quality. Therefore, the load prediction model trained in this application can analyze the load information of each server in the cloud computing platform from both spatial and temporal perspectives, capturing long-term dependencies, thus enabling the trained load prediction model to more accurately predict server load information.

[0061] 2. By performing outlier detection, repair, and normalization (standardization) operations on the original data information, the target data information can be made more accurate, thereby enabling the model to better adapt to different data distributions and changes, and thus improving the model's performance and robustness.

[0062] 3. The embodiments of this application combine the PCA (Principal Components Analysis) algorithm with the LOF (Local Outlier Factor) algorithm, which can more accurately detect outliers and improve robustness.

[0063] 4. The embodiments of this application use a weighted outlier repair method based on GAN (Generative Adversarial Nets) and support vector regression to repair outliers, which can improve the accuracy of outlier repair.

[0064] 5. In the embodiments of this application, during model training or model prediction, the model input includes not only load information but also traffic and / or network performance. By combining historical traffic and / or network performance to predict future load information, the prediction results are more objective and closer to the true future value, thereby further improving the accuracy of load prediction. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0066] Figure 1 A flowchart illustrating a training method for a load prediction model provided in an embodiment of this application;

[0067] Figure 2 This is a schematic diagram of the structure of a load prediction model provided in an embodiment of this application;

[0068] Figure 3 A schematic flowchart illustrating a server load prediction method provided in an embodiment of this application;

[0069] Figure 4 A block diagram illustrating the composition of a training apparatus for a load prediction model provided in an embodiment of this application;

[0070] Figure 5 This is a block diagram of a server load prediction device provided in an embodiment of this application. Detailed Implementation

[0071] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0072] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The terms "comprising" and "having," and any variations thereof, in the embodiments and drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0073] Figure 1 This is a flowchart illustrating a training method for a load prediction model. This method can be applied to electronic or computer equipment and includes the following steps S110-S180. Figure 2As shown, the load prediction model includes an encoder 210 and a decoder 220. The encoder includes a first graph convolutional neural network 211, a first feedforward neural network 212, a first multi-head attention mechanism network 213, and a second feedforward neural network 214. The decoder includes a second graph convolutional neural network 221, a third feedforward neural network 222, a second multi-head attention mechanism network 223, and a fourth feedforward neural network 224.

[0074] The training method for this model will be described in detail below:

[0075] S110: Obtain the training set.

[0076] The training set includes multiple training samples and corresponding ground truth samples for each training sample. Each training sample includes a graph structure representing multiple consecutive historical time points. The corresponding ground truth sample includes a graph structure representing multiple future time points adjacent to the consecutive historical time points. The graph structure at each time point includes the target data information for each of the multiple servers at that time point, and the target data information includes at least load information. The graph structure at each time point can be a topology of multiple servers, where each node represents a server and carries the target data information.

[0077] The duration of multiple consecutive historical moments and the duration of multiple future moments adjacent to them can be the same or different. When the durations are different, the duration of the multiple consecutive historical moments is often greater than the duration of the multiple future moments. For example, if the multiple consecutive historical moments are moments 1-5, the multiple future moments can be moments 6-10 or moments 6-8.

[0078] In addition to load information, target data may also include traffic and / or network performance. Traffic includes inbound traffic, outbound traffic, and total traffic, while load information may include CPU (Central Processing Unit) utilization, memory utilization, etc. Network performance includes latency, bandwidth utilization, etc.

[0079] To improve the stability, generalization performance, and robustness of the load prediction model, after obtaining the raw data information of each server (at least including the raw load information), outlier detection can be performed on the raw data information of each server. When outliers are detected in the raw data information, they are then repaired. Finally, the raw data information after outlier repair can be normalized or standardized to obtain the target data information.

[0080] The method for outlier detection on the raw data of each server includes: dimensionality reduction of the raw data based on the PCA algorithm, and outlier detection using LOF (List of Elements) on the dimensionality-reduced raw data. Specifically, LOF is used to calculate and identify outlier factor scores in the dimensionality-reduced raw data, and outliers are determined based on these scores. By combining PCA and LOF, outlier detection can be more accurate and robustness improved.

[0081] Methods for repairing outliers include: using generative adversarial networks to repair outliers and obtain repaired first data, and using support vector regression algorithms to repair outliers and obtain repaired second data; and performing weighted calculations on the first and second data to obtain the final result after outlier repair.

[0082] The process involves first training a GAN model, taking the original data as input, with the goal of generating synthetic data similar to the original data. After training, a generator is used to generate repaired outlier data; simultaneously, support vector regression is used to generate repaired outlier data. Finally, the repaired data obtained from these two methods are weighted and summed, and the final repaired result is generated based on the weighted sum.

[0083] The embodiments of this application, by detecting and repairing outliers and performing operations such as data normalization or standardization, enable the model to better adapt to different data distributions and changes, thereby improving the model's performance and robustness.

[0084] After preprocessing the raw data to obtain the target data, the length of each data entry can be determined based on the number of servers to be predicted. The data size is then adjusted. The input data refers to all information at a given time point, including traffic, load, and network performance, and each data entry is a graph structure. It is ensured that the input and output tensor sizes of the encoder and decoder in the load prediction model are consistent. These datasets can then be divided into training and test sets. Since this embodiment uses time-series data, the partitioning of the training set differs slightly from typical dataset partitioning. A portion of the data is used as historical data, and a future portion of that historical data is used as the target data. The training set refers to the data used by the load prediction model for learning, while the test set is used to test the model's performance after training. Model performance metrics include accuracy, recall, and mean squared error. Accuracy represents the proportion of correctly predicted samples, recall represents the proportion of correctly predicted positive examples out of all actual positive examples, and mean squared error represents the mean of the squared differences between predicted and true values.

[0085] In addition, to enhance the model input, positional encoding is computed before feeding the training set into the load prediction model. Positional encoding treats each time step (each moment) as an absolute position and uses trigonometric functions (such as sine and cosine functions) to generate periodic codes with different frequencies. Local timestamps used for positional encoding may lead to mismatches between queries and keys between the encoder and decoder. Therefore, in addition to positional encoding, global timestamps are also added as time encoding. This encoding takes into account the influence of date factors such as weekdays, months, and holidays, improving the accuracy of server load prediction.

[0086] S120: Input each graph structure in each training sample into the first graph convolutional neural network to extract the first network spatial features of each graph structure in each training sample; and input each graph structure in each ground value sample into the second graph convolutional neural network to extract the second network spatial features of each graph structure in each ground value sample.

[0087] To enable the load prediction model to extract spatial dependencies, this embodiment incorporates a graph convolutional neural network (GCNN) into the encoder and decoder, performing convolution operations on a graph structure. The convolutional layer filters the input data and extracts features through a set of learnable filters. The core idea of ​​the convolutional layer is weight sharing; all filters compute the same convolution on the input data, thus reducing the number of parameters that need to be trained. The convolution operation can be viewed as a special type of linear transformation that converts local relationships in the input data into global information. It also possesses translation invariance, meaning the feature extraction result is independent of the location of the input data. This makes convolutional layers highly suitable for processing spatially and temporally adjacent data, capturing their inherent patterns and structures, and improving the model's robustness.

[0088] S130: Each first road network spatial feature is processed by convolution calculation of the first feedforward neural network to obtain the first road network spatial feature after convolution calculation, and each second road network spatial feature is processed by convolution calculation of the third feedforward neural network to obtain the second road network spatial feature after convolution calculation.

[0089] A feedforward neural network is an artificial neural network whose structure consists of multiple layers of nodes that transmit information in a specific direction. Inputting the first and second network spatial features into the feedforward neural network respectively can refine these features.

[0090] S140: Add corresponding temporal information to the first network spatial features calculated by convolution at each time step, and input the first network spatial features of each training sample with added temporal information into the first multi-head attention mechanism network to obtain the first network spatiotemporal features of each training sample.

[0091] In this context, the temporal information corresponding to the first road network spatial feature calculated by convolution at each time step is the time step in which the first road network spatial feature calculated by convolution is located. After adding the corresponding temporal information to the first road network spatial features calculated by convolution at each time step in each training sample, all the first road network spatial features in a training sample can be input into the first multi-head attention mechanism network to ensure that the model can consider the temporal dynamics of the road network features and obtain the first road network time step features of each training sample.

[0092] S150: The first network spatiotemporal features of each training sample are processed by convolution of the second feedforward neural network to obtain the first network spatiotemporal features of each training sample after convolution.

[0093] To refine the spatiotemporal features of the first network, the spatiotemporal features of the first network for each training sample can be processed by convolution of the second feedforward neural network to obtain the spatiotemporal features of the first network after convolution of each training sample.

[0094] S160: Input the first network spatiotemporal features calculated by convolution, and the multiple second network spatial features in the corresponding ground truth samples after adding time information, into the second multi-head attention mechanism network to obtain the third network spatiotemporal features of multiple future time points corresponding to each training sample, as well as the second network spatiotemporal features of each ground truth sample.

[0095] After the encoder outputs the first network spatiotemporal features calculated by convolution, it can be input into the second multi-head attention mechanism network of the decoder for decoding calculation to obtain the third network spatiotemporal features of multiple future time steps corresponding to each training sample. At the same time, after adding the corresponding time information to the second network spatial features calculated by convolution at each time step, the second network spatial features of each ground truth sample with the added time information are input into the second multi-head attention mechanism network to obtain the second network spatiotemporal features of each ground truth sample.

[0096] S170: Input the third network spatiotemporal features of multiple future time points corresponding to each training sample, and the second network spatiotemporal features of each ground value sample into the fourth feedforward neural network to obtain the third network spatiotemporal features and the second network spatiotemporal features after convolution calculation.

[0097] To refine the spatiotemporal features of the third and second network networks, the spatiotemporal features of the third and second network networks can be input into the fourth feedforward neural network to obtain the spatiotemporal features of the third and second network networks after convolution calculation.

[0098] S180: Calculate the current loss value based on the spatiotemporal features of the third road network after convolution and the spatiotemporal features of the second road network after convolution. If the current loss value does not meet the convergence condition, adjust the model parameters of the load prediction model and continue to train the load prediction model with adjusted parameters until the current loss value meets the convergence condition, and obtain the final load prediction model.

[0099] After obtaining the spatiotemporal features of the third network and the second network after convolution, the current loss value can be calculated based on the difference between the spatiotemporal features of the third network and the second network. For example, for each pair of training samples and ground truth samples, the difference between the spatiotemporal features of the third network and the corresponding spatiotemporal features of the second network after convolution can be calculated, and then the average of these differences can be used as the current loss value.

[0100] When the current loss value is greater than or equal to the loss threshold, it is determined that the current loss value does not meet the convergence condition. At this time, the model parameters of the load prediction model can be adjusted. Then, based on the load prediction model with adjusted parameters, steps S110-S170 are continued until the current loss value is less than the loss threshold. At this time, it is determined that the current loss value meets the convergence condition, and the load prediction model obtained by training is taken as the final load prediction model.

[0101] In one implementation, the loss function used during model training is the cross-entropy loss function. The cross-entropy loss function measures the probability distribution of target data information at each future time step and compares the predicted probabilities with the actual target data information. Simultaneously, to avoid overfitting, a regularization term, such as L2 regularization, is typically added.

[0102] After obtaining a converged load prediction model, the test dataset is input into the load prediction model, the output of the model is visualized or analyzed, its performance indicators are calculated and evaluated.

[0103] It should be added that the embodiments of this application can perform residual connections between the networks of the encoder and decoder, and perform layer normalization operation after the residual connections, which alleviates the gradient vanishing problem caused by increasing the depth in deep neural networks. Furthermore, Dropout operation is used, which discards the output of some neurons in the network to reduce overfitting.

[0104] Similar to the encoder, the decoder incorporates a multi-head attention mechanism for interacting with the encoder's output. The multi-head attention layer treats each position in the input sequence as a query (Q), key (K), and value (V), calculating the attention distribution between each position and all positions, resulting in a weighted sum representing the contextual information at that position. The fully connected layer then performs forward propagation on this contextual information, yielding its output. Specifically, the calculation formula for multi-head attention is as follows:

[0105]

[0106] Where, d k Given a vector dimension, Q, K, and V are all vectors. This calculation formula shows that for each query Q, Multi-Head Attention performs a weighted sum of all values ​​V based on its similarity to all keys K.

[0107] The second multi-head attention mechanism network in the decoder is similar to the first multi-head attention mechanism network in the encoder. However, when calculating the attention distribution, the second multi-head attention mechanism network in the decoder only considers the positions before the current position, thus avoiding the problem of using future information in the decoder.

[0108] The load prediction model training method provided in this application includes an encoder and a decoder, both of which contain a graph convolutional neural network, a feedforward neural network, and a multi-head attention mechanism network. Each graph structure in each training sample sequentially passes through the first graph convolutional neural network, the first feedforward neural network, the first multi-head attention mechanism network, and the second feedforward neural network in the encoder, ultimately outputting the first network spatiotemporal features after convolutional calculation for each training sample. Each graph structure in each ground truth sample is sequentially input into the second graph convolutional neural network and the third feedforward neural network to obtain the second network spatial features after convolutional calculation. Each training sample... The third network spatiotemporal features corresponding to multiple future time points, and the second network spatiotemporal features of each ground truth sample are input into a fourth feedforward neural network to obtain the convolutionally calculated third network spatiotemporal features and the convolutionally calculated second network spatiotemporal features. Based on the convolutionally calculated third network spatiotemporal features and the convolutionally calculated second network spatiotemporal features, the current loss value is calculated. If the current loss value does not meet the convergence condition, the model parameters of the load prediction model are adjusted, and the load prediction model with adjusted parameters is trained again until the current loss value meets the convergence condition, thus obtaining the final load prediction model for load prediction. Therefore, the embodiments of this application can analyze the load information of each server in the cloud computing platform from both spatial and temporal perspectives, capture long-term dependencies, and enable the trained load prediction model to more accurately predict the server load information.

[0109] Based on the above method embodiments, another embodiment of this application provides a server load prediction method, such as... Figure 3 As shown, the method includes:

[0110] S310: Obtain target data information of each server in the cloud computing platform at multiple recent historical moments.

[0111] The target data information includes at least load information, and may also include traffic and / or network performance. Traffic includes inbound traffic, outbound traffic, and total traffic, while load information may include CPU utilization, memory utilization, etc. Network performance includes latency, bandwidth utilization, etc.

[0112] When it is necessary to predict the load information of each server at multiple future moments, target data information from multiple recent historical moments can be obtained first, such as target data information from multiple moments within the last 10 minutes. The size of a moment can be determined according to actual needs; for example, a moment can refer to 1 minute or 5 minutes.

[0113] To improve the stability, generalization performance, and robustness of the load prediction model, after obtaining the raw data information of each server from multiple recent historical moments, outlier detection can be performed on the raw data information of each server. When outliers are detected in the raw data information, they are then repaired. Finally, the raw data information after outlier repair can be normalized or standardized to obtain the target data information.

[0114] The method for outlier detection on the raw data of each server includes: dimensionality reduction of the raw data based on the PCA algorithm, and outlier detection using LOF (List of Elements) on the dimensionality-reduced raw data. Specifically, LOF is used to calculate and identify outlier factor scores in the dimensionality-reduced raw data, and outliers are determined based on these scores. By combining PCA and LOF, outlier detection can be more accurate and robustness improved.

[0115] Methods for repairing outliers include: using generative adversarial networks to repair outliers and obtain repaired first data, and using support vector regression algorithms to repair outliers and obtain repaired second data; and performing weighted calculations on the first and second data to obtain the final result after outlier repair.

[0116] The process involves first training a GAN model, taking the original data as input, with the goal of generating synthetic data similar to the original data. After training, a generator is used to generate repaired outlier data; simultaneously, support vector regression is used to generate repaired outlier data. Finally, the repaired data obtained from these two methods are weighted and summed, and the final repaired result is generated based on the weighted sum.

[0117] S320: For each historical moment, a graph structure is generated based on the target data information of each server in the cloud computing platform.

[0118] After obtaining the target data information of each server in the cloud computing platform, a graph structure for each historical moment can be generated according to the topology of each server, so that each node in the graph structure represents a server, and the information carried by each node includes the target data information of the server at that historical moment.

[0119] S330: Input the graph structure of multiple recent historical moments into the load prediction model to predict the load information of each server at multiple future moments.

[0120] The load prediction model is trained according to the training method of the load prediction model provided in any of the above embodiments. By inputting the graph structure of multiple recent historical moments into the pre-trained load prediction model, and through the processing of the encoder and decoder in the load prediction model, the load information of each server at multiple future moments can be predicted.

[0121] The server load prediction method provided in this application first acquires target data information of each server in a cloud computing platform at multiple recent historical moments. Then, for each historical moment, a graph structure is generated based on the target data information of each server in the cloud computing platform. Finally, the graph structure of multiple recent historical moments is input into the load prediction model to predict the load information of each server at multiple future moments. The load prediction model includes an encoder and a decoder, and both the encoder and decoder contain graph convolutional neural networks, feedforward neural networks, and multi-head attention mechanisms. In this application embodiment, the target data information is first transformed into a graph structure including the server topology. Then, based on the graph convolutional neural network, road network spatial features are extracted from the spatial dimension. Then, based on the multi-head attention mechanism network, the dependencies of each road network spatial feature are analyzed from the temporal dimension to extract the spatiotemporal features of the road network. After each feature extraction, the extracted features are refined through a feedforward neural network, which can accelerate the convergence of the load prediction model and improve the quality of the load prediction model. Therefore, the load prediction model trained in this application embodiment can analyze the load information of each server in the cloud computing platform from both spatial and temporal perspectives, and can capture long-term dependencies, thereby enabling the trained load prediction model to more accurately predict the server load information.

[0122] Based on the above method embodiments, another embodiment of this application provides a training device for a load prediction model. The load prediction model includes an encoder and a decoder. The encoder includes a first graph convolutional neural network, a first feedforward neural network, a first multi-head attention mechanism network, and a second feedforward neural network. The decoder includes a second graph convolutional neural network, a third feedforward neural network, a second multi-head attention mechanism network, and a fourth feedforward neural network. Figure 4 As shown, the device includes:

[0123] The acquisition unit 410 is used to acquire a training set, wherein the training set includes multiple training samples and ground value samples corresponding to each training sample, each training sample includes a graph structure of multiple consecutive historical moments, the ground value samples corresponding to each training sample include a graph structure of multiple future moments adjacent to the multiple consecutive historical moments, and the graph structure of each moment includes target data information of each of the multiple servers at that moment, the target data information including at least load information.

[0124] The first feature extraction unit 420 is used to input each graph structure in each training sample into the first graph convolutional neural network to extract the first road network spatial features of each graph structure in each training sample.

[0125] The second feature extraction unit 430 is used to input each graph structure in each ground value sample into the second graph convolutional neural network to extract the second road network spatial features of each graph structure in each ground value sample.

[0126] The first convolution calculation unit 440 is used to perform convolution calculation on each of the first road network spatial features through the first feedforward neural network to obtain the first road network spatial features after convolution calculation.

[0127] The second convolution calculation unit 450 is used to perform convolution calculation on each of the second road network spatial features through the third feedforward neural network to obtain the second road network spatial features after convolution calculation.

[0128] Add unit 460 to add corresponding temporal information to the first network spatial features after convolution calculation at each time step;

[0129] The first attention processing unit 470 is used to input the first road network spatial features of each training sample after adding time information into the first multi-head attention mechanism network to obtain the first road network spatiotemporal features of each training sample.

[0130] The third convolutional computation unit 480 is used to perform convolutional computation on the first network spatiotemporal features of each training sample through the second feedforward neural network to obtain the first network spatiotemporal features of each training sample after convolutional computation.

[0131] The second attention processing unit 490 is used to input the first network spatiotemporal features after convolution calculation and multiple second network spatial features in the corresponding ground value samples after adding time information into the second multi-head attention mechanism network to obtain the third network spatiotemporal features of multiple future times corresponding to each training sample, and the second network spatiotemporal features of each ground value sample.

[0132] The fourth convolutional computation unit 4100 is used to input the third network spatiotemporal features of multiple future time points corresponding to each training sample, and the second network spatiotemporal features of each ground value sample into the fourth feedforward neural network to obtain the third network spatiotemporal features and the second network spatiotemporal features after convolutional computation.

[0133] The training unit 4110 is used to calculate the current loss value based on the spatiotemporal features of the third network after convolution and the spatiotemporal features of the second network after convolution. When the current loss value does not meet the convergence condition, the model parameters of the load prediction model are adjusted, and the load prediction model with adjusted parameters is trained until the current loss value meets the convergence condition, thus obtaining the final load prediction model.

[0134] In one possible implementation, the acquisition unit 410 includes:

[0135] An outlier detection module is used to detect outliers in the raw data information of each server.

[0136] The repair module is used to repair the abnormal values ​​when an abnormal value is detected in the original data information.

[0137] The processing module is used to normalize or standardize the original data information after outlier repair to obtain the target data information.

[0138] In one possible implementation, the outlier detection module is used to perform dimensionality reduction processing on the original data information based on the principal component analysis (PCA) algorithm, and to perform outlier detection on the dimensionality-reduced original data information using the local outlier factor (LOF).

[0139] In one possible implementation, the repair module is used to repair the outliers using a generative adversarial network to obtain repaired first data, and to repair the outliers using a support vector regression algorithm to obtain repaired second data; the first data and the second data are weighted and calculated to obtain the final result after the outlier repair.

[0140] In one possible implementation, the target data information further includes: traffic and / or network performance.

[0141] The load prediction model training device provided in this application includes an encoder and a decoder, both of which contain a graph convolutional neural network, a feedforward neural network, and a multi-head attention mechanism network. Each graph structure in each training sample sequentially passes through the first graph convolutional neural network, the first feedforward neural network, the first multi-head attention mechanism network, and the second feedforward neural network in the encoder, ultimately outputting the first network spatiotemporal features after convolutional calculation for each training sample. Each graph structure in each ground truth sample is sequentially input into the second graph convolutional neural network and the third feedforward neural network to obtain the second network spatial features after convolutional calculation. Each training sample... The third network spatiotemporal features corresponding to multiple future time points, and the second network spatiotemporal features of each ground truth sample are input into a fourth feedforward neural network to obtain the convolutionally calculated third network spatiotemporal features and the convolutionally calculated second network spatiotemporal features. Based on the convolutionally calculated third network spatiotemporal features and the convolutionally calculated second network spatiotemporal features, the current loss value is calculated. If the current loss value does not meet the convergence condition, the model parameters of the load prediction model are adjusted, and the load prediction model with adjusted parameters is trained again until the current loss value meets the convergence condition, thus obtaining the final load prediction model for load prediction. Therefore, the embodiments of this application can analyze the load information of each server in the cloud computing platform from both spatial and temporal perspectives, capture long-term dependencies, and enable the trained load prediction model to more accurately predict the server load information.

[0142] Based on the above method embodiments, another embodiment of this application provides a server load prediction device, such as... Figure 5 As shown, the device includes:

[0143] The acquisition unit 510 is used to acquire target data information of each server in the cloud computing platform at multiple recent historical moments, and the target data information includes at least load information.

[0144] The generation unit 520 is used to generate a graph structure for each historical moment based on the target data information of each server in the cloud computing platform.

[0145] The prediction unit 530 is used to input the graph structure of multiple recent historical moments into the load prediction model to predict the load information of each server at multiple future moments, wherein the load prediction model is trained according to the training method of the load prediction model provided in any of the above embodiments.

[0146] The server load prediction device provided in this application embodiment can first acquire target data information of each server in the cloud computing platform at multiple recent historical moments, then generate a graph structure for each historical moment based on the target data information of each server in the cloud computing platform, and finally input the graph structure of multiple recent historical moments into the load prediction model to predict the load information of each server at multiple future moments. The load prediction model includes an encoder and a decoder, and both the encoder and decoder include graph convolutional neural networks, feedforward neural networks, and multi-head attention mechanism networks. In this application embodiment, the target data information is first transformed into a graph structure including the server topology, then the road network spatial features are extracted from the spatial dimension based on the graph convolutional neural network, and then the dependencies of each road network spatial feature are analyzed from the temporal dimension based on the multi-head attention mechanism network to extract the spatiotemporal features of the road network. After each feature extraction, the extracted features are refined by the feedforward neural network, which can accelerate the convergence of the load prediction model and improve the quality of the load prediction model. Therefore, the load prediction model trained in this application embodiment can analyze the load information of each server in the cloud computing platform from both spatial and temporal perspectives, and can capture long-term dependencies, thereby enabling the trained load prediction model to more accurately predict the server load information.

[0147] Based on the above method embodiments, another embodiment of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the above embodiments.

[0148] Based on the above method embodiments, another embodiment of this application provides an electronic device or computer device, including:

[0149] One or more processors;

[0150] The processor is coupled to a storage device for storing one or more programs;

[0151] When the one or more programs are executed by the one or more processors, the electronic device or computer device performs the method as described in any of the above embodiments.

[0152] Based on the above embodiments, another embodiment of this application provides a computer program product, which includes instructions that, when executed on a computer or processor, cause the computer or processor to perform the method described in any of the above embodiments.

[0153] The above-described apparatus embodiments correspond to the method embodiments and have the same technical effects. For detailed descriptions, please refer to the method embodiments. The apparatus embodiments are derived from the method embodiments; detailed descriptions can be found in the method embodiments section, and will not be repeated here. Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application.

[0154] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A training method for a load prediction model, characterized in that, The load prediction model includes an encoder and a decoder. The encoder includes a first graph convolutional neural network, a first feedforward neural network, a first multi-head attention mechanism network, and a second feedforward neural network. The decoder includes a second graph convolutional neural network, a third feedforward neural network, a second multi-head attention mechanism network, and a fourth feedforward neural network. The method includes: Obtain a training set, wherein the training set includes multiple training samples and ground value samples corresponding to each training sample, each training sample includes a graph structure of multiple consecutive historical moments, the ground value samples corresponding to each training sample include a graph structure of multiple future moments adjacent to the multiple consecutive historical moments, the graph structure of each moment includes the target data information of each of the multiple servers at that moment, the graph structure of each moment is the topology of the multiple servers, each node in the graph structure represents a server, each node carries the target data information, the target data information includes at least load information, the load information includes CPU utilization and / or memory utilization; Each graph structure in each training sample is input into the first graph convolutional neural network to extract the first road network spatial features of each graph structure in each training sample; and each graph structure in each ground value sample is input into the second graph convolutional neural network to extract the second road network spatial features of each graph structure in each ground value sample. Each of the first road network spatial features is processed by the first feedforward neural network to obtain the first road network spatial features after convolution calculation, and each of the second road network spatial features is processed by the third feedforward neural network to obtain the second road network spatial features after convolution calculation. Add corresponding temporal information to the first network spatial features calculated by convolution at each time step, and input the first network spatial features of each training sample with added temporal information into the first multi-head attention mechanism network to obtain the first network spatiotemporal features of each training sample; The first network spatiotemporal features of each training sample are processed by convolution of the second feedforward neural network to obtain the first network spatiotemporal features of each training sample after convolution. The first network spatiotemporal features calculated by convolution, and multiple second network spatial features in the corresponding ground truth samples after adding time information, are input into the second multi-head attention mechanism network to obtain the third network spatiotemporal features of multiple future time moments corresponding to each training sample, and the second network spatiotemporal features of each ground truth sample. The third network spatiotemporal features corresponding to multiple future time points for each training sample, and the second network spatiotemporal features for each ground value sample are input into the fourth feedforward neural network to obtain the third network spatiotemporal features and the second network spatiotemporal features after convolution calculation. Based on the spatiotemporal features of the third network after convolution and the spatiotemporal features of the second network after convolution, the current loss value is calculated. If the current loss value does not meet the convergence condition, the model parameters of the load prediction model are adjusted, and the load prediction model with adjusted parameters is trained until the current loss value meets the convergence condition, thus obtaining the final load prediction model.

2. The method according to claim 1, characterized in that, The method for obtaining target data information for each of the servers includes: Anomaly detection is performed on the raw data information of each of the servers; When an outlier is detected in the original data, the outlier is repaired. The original data information after outlier repair is normalized or standardized to obtain the target data information.

3. The method according to claim 2, characterized in that, The outlier detection of the raw data information of each of the servers includes: The original data information is dimensionality reduced based on the principal component analysis (PCA) algorithm, and outlier detection is performed on the dimensionality-reduced original data information using the local outlier factor (LOF).

4. The method according to claim 2, characterized in that, The repair of the outlier includes: Generative adversarial networks are used to repair the outliers to obtain the first data after repair, and support vector regression algorithms are used to repair the outliers to obtain the second data after repair. The first data and the second data are weighted and calculated to obtain the final result after the outlier is repaired.

5. The method according to any one of claims 1-4, characterized in that, The target data information also includes: traffic and / or network performance.

6. A method for predicting server load, characterized in that, The method includes: Obtain target data information of each server in the cloud computing platform at multiple recent historical moments, wherein the target data information includes at least load information; For each historical moment, a graph structure is generated based on the target data information of each server in the cloud computing platform; The load prediction model is input into the graph structure of the most recent historical moments to predict the load information of each server at multiple future moments, wherein the load prediction model is trained by the method according to any one of claims 1-5.

7. A training device for a load prediction model, characterized in that, The load prediction model includes an encoder and a decoder. The encoder includes a first graph convolutional neural network, a first feedforward neural network, a first multi-head attention mechanism network, and a second feedforward neural network. The decoder includes a second graph convolutional neural network, a third feedforward neural network, a second multi-head attention mechanism network, and a fourth feedforward neural network. The device includes: An acquisition unit is used to acquire a training set, wherein the training set includes multiple training samples and ground value samples corresponding to each training sample, each training sample includes a graph structure of multiple consecutive historical moments, the ground value samples corresponding to each training sample include a graph structure of multiple future moments adjacent to the multiple consecutive historical moments, the graph structure of each moment includes target data information of each of the multiple servers at that moment, the graph structure of each moment is a topology structure of multiple servers, each node in the graph structure represents a server, each node carries the target data information, the target data information includes at least load information, the load information includes CPU utilization and / or memory utilization; The first feature extraction unit is used to input each graph structure in each training sample into the first graph convolutional neural network to extract the first road network spatial features of each graph structure in each training sample. The second feature extraction unit is used to input each graph structure in each ground value sample into the second graph convolutional neural network to extract the second road network spatial features of each graph structure in each ground value sample; The first convolutional computation unit is used to perform convolutional computation on each of the first road network spatial features through the first feedforward neural network to obtain the first road network spatial features after convolutional computation. The second convolutional computation unit is used to perform convolutional computation on each of the second network spatial features through the third feedforward neural network to obtain the second network spatial features after convolutional computation. Add units to add corresponding temporal information to the first network spatial features after convolution calculation at each time step; The first attention processing unit is used to input the first road network spatial features of each training sample after adding time information into the first multi-head attention mechanism network to obtain the first road network spatiotemporal features of each training sample. The third convolutional computation unit is used to perform convolutional computation on the first network spatiotemporal features of each training sample through the second feedforward neural network to obtain the first network spatiotemporal features of each training sample after convolutional computation. The second attention processing unit is used to input the first network spatiotemporal features after convolution calculation and multiple second network spatial features in the corresponding ground value samples after adding time information into the second multi-head attention mechanism network to obtain the third network spatiotemporal features of multiple future times corresponding to each training sample, and the second network spatiotemporal features of each ground value sample. The fourth convolutional computation unit is used to input the third network spatiotemporal features of multiple future time points corresponding to each training sample, and the second network spatiotemporal features of each ground value sample into the fourth feedforward neural network to obtain the third network spatiotemporal features and the second network spatiotemporal features after convolutional computation. The training unit is adjusted to calculate the current loss value based on the spatiotemporal features of the third network after convolution and the spatiotemporal features of the second network after convolution. When the current loss value does not meet the convergence condition, the model parameters of the load prediction model are adjusted, and the load prediction model with adjusted parameters is trained until the current loss value meets the convergence condition, thus obtaining the final load prediction model.

8. A load prediction device for a server, characterized in that, The device includes: The acquisition unit is used to acquire target data information of each server in the cloud computing platform at multiple recent historical moments, wherein the target data information includes at least load information. A generation unit is used to generate a graph structure for each historical moment based on the target data information of each server in the cloud computing platform; A prediction unit is used to input the graph structure of multiple recent historical moments into a load prediction model to predict the load information of each server at multiple future moments, wherein the load prediction model is trained by the method according to any one of claims 1-5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the method as described in any one of claims 1-5, or implements the method as described in claim 6.

10. An electronic device, characterized in that, The electronic device includes: One or more processors; The processor is coupled to a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the electronic device causes the electronic device to implement the method as described in any one of claims 1-5, or to implement the method as described in claim 6.

Images