A cloud service workload prediction method and system based on convolution enhanced Transformer

Abstract

The application relates to a cloud service workload prediction method and system based on a convolution enhanced Transformer, which comprises the following steps: collecting cloud server workload data and preprocessing the cloud server workload data to decompose the cloud server workload data into a trend component and a residual component; constructing a workload prediction model, training the workload prediction model by using the trend component and the residual component, and obtaining a trained workload prediction model; predicting the cloud server workload by using the trained workload prediction model to obtain a workload prediction value; predicting the trend component by using a trend information capturing module to obtain a prediction result of the trend component; predicting the residual component by using a convolution enhanced Transformer encoder module to obtain a prediction result of the residual component; and fusing the prediction result of the trend component and the prediction result of the residual component by using a feature fusion module to obtain a final workload prediction value.

Description

Technical Field

[0001] This invention relates to a cloud service workload prediction method and system based on convolutional augmented Transformer, belonging to the field of cloud computing technology. Background Technology

[0002] With the popularization of cloud computing and cloud-native technologies, microservice architecture has been widely adopted. In such environments, dynamically and efficiently allocating computing resources according to business load is key to ensuring quality of service (QoS) and controlling costs.

[0003] Currently, most mainstream resource scaling technologies, such as the default horizontal Pod Autoscaler (HPA) on the Kubernetes platform, adopt a reactive strategy. This strategy monitors real-time resource utilization (such as CPU and memory) and only triggers scaling operations when the metrics exceed preset thresholds. The drawback of this method is its inherent latency: when the load spikes suddenly, the system is already under high pressure, and scaling up at this point is too late, easily leading to prolonged service response times or even service level agreement (SLA) violations.

[0004] To address the lag issue in responsive strategies, academia and industry have begun researching proactive scaling techniques, the core of which is to predict future workloads in advance and allocate resources based on the predictions. Existing prediction methods primarily rely on recurrent neural networks (RNNs) and their variants, such as Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs). However, these models suffer from long-term memory decay and low training efficiency when handling long sequences.

[0005] In recent years, Transformer models, with their self-attention mechanism at their core, have demonstrated great potential in time series forecasting because they can effectively capture long-term dependencies in sequences. However, standard Transformer models have limited ability to extract local features of time series, such as short-term load spikes or jitter. Furthermore, real-world workload data in cloud environments often suffers from distribution shift, meaning the distribution of training data differs from the distribution of actual online data, which can severely impact the accuracy and stability of prediction models.

[0006] Therefore, designing a high-precision prediction model that can simultaneously capture the global trends and local fluctuations of workloads and effectively address the problem of data distribution offset is a technical challenge that urgently needs to be solved in the field of cloud-native resource management. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a cloud service workload prediction method and system based on convolutional enhanced Transformer. The method first processes the input historical time series of workloads using a reversible instance normalization module to enhance the model's robustness to changes in data distribution. Then, a moving average kernel is used to decompose the normalized sequence into a trend component representing long-term stable changes and a residual component containing short-term sharp fluctuations; differentiated processing strategies are applied to the two components. For the trend component with relatively simple change patterns, a linear layer is used for fitting and prediction. For the residual component containing complex nonlinear relationships, this invention designs a novel convolutional enhanced Transformer encoder for deep feature extraction. The core of this encoder lies in using a self-attention mechanism in parallel to capture the global dependencies of the sequence, while simultaneously using a convolutional neural network module to extract the local feature patterns of the sequence. Global and local features are effectively fused through residual connections and other methods to form a more comprehensive representation of the workload.

[0008] In the prediction generation stage, this invention uses a lightweight multilayer perceptron (MLP) instead of a traditional Transformer decoder to map the deep features extracted by the encoder into predictions for the residual components, thereby significantly reducing computational complexity and inference latency. Finally, the predictions for the trend components and the residual components are weighted and synthesized using learnable weights, and then restored to the original data scale through the inverse operation of the reversible instance normalization module, resulting in the final high-precision prediction of future workloads.

[0009] Terminology Explanation: 1. Position-wise Feed-Forward Network: This is an important component of the Transformer architecture, typically located after the self-attention layer. It is a position-wise feed-forward neural network where the feature vector at each position undergoes the same processing, but the outputs at different positions are independent. The role of the position-wise feed-forward network is to further process the features output from the self-attention layer through a series of non-linear transformations. This network includes two main linear transformations and an activation function (usually ReLU). Its core idea is to process the representation at each position independently, preserving the "position-independent" property of the Transformer.

[0010] 2. Self-Attention: A core component of the standard Transformer, it captures global, long-distance dependencies by calculating the correlation score between any two points in time in a sequence. For example, it can discover similar patterns in load peaks at 9 AM and 3 PM each day, even if they are far apart in time series.

[0011] The technical solution of the present invention is as follows: The first aspect of this invention provides a cloud service workload prediction method based on convolutional augmented Transformer, comprising: Step 1: Collect cloud server workload data and preprocess it, decomposing it into trend components and residual components; Step 2: Build a load prediction model, train it using trend components and residual components to obtain a trained load prediction model; use the trained load prediction model to predict the workload of the cloud server to obtain the predicted workload value. The load prediction model includes a trend information capture module, a convolutional enhanced Transformer encoder module, and a feature fusion module; The trend component is predicted by the trend information capture module, and the prediction result of the trend component is obtained. The residual components are predicted by the convolution-enhanced Transformer encoder module to obtain the prediction results of the residual components; The feature fusion module fuses the prediction results of the trend component and the prediction results of the residual component to obtain the final workload prediction value.

[0012] According to a preferred embodiment of the present invention, cloud server workload data is collected and preprocessed to decompose it into trend components and residual components; including: Collect cloud server workload data, including CPU utilization, memory usage, network I / O, and requests per second (QPS). The cloud server workload data is adjusted to a two-dimensional tensor X of (T, N), where T represents the length of the time series and N represents the total dimension of the features; Calculate the mean μ_i and standard deviation σ_i of each feature of the two-dimensional tensor X on the time axis T, where i∈[1,N]; Normalize each feature as follows: X_norm_i = (X_i - μ_i) / σ_i; Where X_norm_i represents the normalized sequence of the i-th feature, and X_i represents the two-dimensional tensor of the i-th feature; the normalized sequence X_norm is obtained by normalizing the N features; The normalized sequence X_norm is decomposed into two parts: long-term trend and short-term residuals. This includes setting a sliding window of size k and sliding it on the time axis of X_norm, calculating the average of all data within each sliding window, forming a new sequence, and obtaining the trend component X_tre. The residual component X_res is obtained by subtracting X_res element by element, X_res = X_norm - X_tre.

[0013] According to a preferred embodiment of the present invention, the trend component is predicted by a trend information capture module to obtain the prediction result of the trend component; including: The trend information capture module includes a fully connected layer. The trend component X_tre is fed into the fully connected layer to learn the linear mapping relationship and output the prediction result X_tre_out of the trend component.

[0014] According to a preferred embodiment of the present invention, the residual components are predicted using a convolution-enhanced Transformer encoder module to obtain the prediction results of the residual components; including: The convolution-enhanced Transformer encoder module includes an embedding layer, a multi-head attention module, a convolutional module, a feedforward network module, and a decoder module; The residual component X_res is mapped from the N-dimensional feature space to a higher-dimensional feature representation through the embedding layer, resulting in X_input; The output X_input of the embedding layer is simultaneously input into the multi-head attention module and the convolution module for parallel processing. Among them, the multi-head attention module calculates the correlation score between any two time points to capture global dependencies; Convolutional modules extract local features from time series data; The outputs of the multi-head attention module and the convolution module are added and fused, and the result is input into the feedforward network module; The result of the summation and fusion is nonlinearly transformed by the feedforward network module. The result of the nonlinear transformation is added to the result of the summation and fusion, and then decoded by the decoder module to finally obtain the prediction result of the residual component.

[0015] More preferably, the multi-head attention module includes multi-head attention operation, first residual connection and first layer normalization; The convolution module includes convolution operations, a second residual connection, and a second normalization layer; The feedforward network module includes a position feedforward network, a third residual connection, and a third-layer normalization; First, in the multi-head attention module, the output X_input of the embedding layer undergoes multi-head attention operation, is then connected to X_input via a first residual, and the calculation result is normalized in the first layer to obtain the output X_att, as shown below: X_att = LayerNorm(X_input + MultiheadAttention(X_input)); Where X_att represents the output of the multi-head attention module, LayerNorm represents the layer normalization calculation of the first normalization layer, and MultiheadAttention represents the multi-head self-attention calculation of the multi-head attention operation; In the convolution module, the output X_input of the embedding layer undergoes a convolution operation followed by a second residual connection. The computation result is then processed through a second-layer normalization to obtain the output X_conv, as shown below: X_conv = LayerNorm(X_input + Convolution(X_input)); Where X_conv represents the output of the convolution module, LayerNorm represents layer normalization, and Convolution represents the convolution operation; the convolution operation includes point convolution and depthwise convolution. Point convolution expands the number of input channels through one-dimensional convolution and uses the GLU activation function to reduce the number of channels back to the original number; depthwise convolution extracts features from each channel through convolution calculation. Then, the outputs of the multi-head attention module and the convolutional module are added and fused to obtain the enhanced feature X_hid, which includes global context information and local detail information, as shown below: X_hid = X_att + X_conv; The enhanced feature X_hid is input into the feedforward network module, and the enhanced feature X_hid is nonlinearly transformed through the position feedforward network. The result after nonlinear transformation is connected to the enhanced feature X_hid through a third residual connection, and then the encoded output is obtained through a third layer of normalization; Finally, the encoded output is sent to the decoder module, which uses a multilayer perceptron to decode the encoded output and finally outputs the prediction result X_res_out of the residual component.

[0016] According to a preferred embodiment of the present invention, the prediction results of the trend component and the prediction results of the residual component are fused by a feature fusion module to obtain the final workload prediction value; including: The prediction results X_tre_out of the trend component and X_res_out of the residual component are fused together as follows: ; in, This represents the result after feature fusion, where W is a learnable parameter; Using the mean μ_i and standard deviation σ_i from the preprocessing step Performing the reverse operation yields the final prediction result, as shown below: Y = * σ_i +μ_i; Where Y represents the final prediction result.

[0017] A computer device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a cloud service workload prediction method based on convolutional augmented Transformer.

[0018] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a cloud service workload prediction method based on convolutional augmented Transformer.

[0019] A second aspect of the present invention provides a cloud service workload prediction system based on convolutional augmented Transformer, comprising: The preprocessing module is configured to: collect cloud server workload data and preprocess it, decomposing it into trend components and residual components; The model prediction module is configured to: build a load prediction model, train it using trend components and residual components to obtain a trained load prediction model; and use the trained load prediction model to predict the workload of cloud servers to obtain the predicted workload value. The load prediction model includes a trend information capture module, a convolutional enhanced Transformer encoder module, and a feature fusion module; The trend component is predicted by the trend information capture module, and the prediction result of the trend component is obtained. The residual components are predicted by the convolution-enhanced Transformer encoder module to obtain the prediction results of the residual components; The feature fusion module fuses the prediction results of the trend component and the prediction results of the residual component to obtain the final workload prediction value.

[0020] The beneficial effects of this invention are as follows: 1. Higher prediction accuracy: By decomposing time series into trend and residual components and combining the global modeling capability of Transformer with the local feature extraction capability of convolution, this invention can more accurately fit complex workload patterns, especially when dealing with mixed workloads containing long-term trends and short-term spikes, the prediction error is significantly reduced.

[0021] 2. Enhanced robustness: The innovative introduction of the Reversible Instance Normalization (RevIN) mechanism, which calculates the mean and standard layer and normalizes them, and then uses the same mean and standard deviation to perform the reverse operation on the feature fusion result, effectively solves the data distribution offset problem that is common in cloud environments, making the model more stable and reliable when facing real and variable online loads.

[0022] 3. Higher computational efficiency: The lightweight multilayer perceptron (MLP) is used to replace the computationally intensive Transformer decoder. While ensuring prediction performance, the inference latency of the model is significantly reduced, making it more suitable for real-time autoscaling scenarios that require fast response.

[0023] 4. More reasonable model design: A differentiated processing strategy is adopted, using models of different complexities to learn data components with different characteristics (trends and residuals), which conforms to the principles of information theory, avoids the waste of model capabilities, and improves the overall learning efficiency. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the cloud service workload prediction model based on convolutional augmented Transformer of the present invention. Figure 2 This is a flowchart of the cloud service workload prediction process based on convolutional enhanced Transformer of the present invention. Detailed Implementation

[0025] The present invention will be further described below with reference to the embodiments and accompanying drawings, but is not limited thereto.

[0026] Example 1 A cloud service workload prediction method based on convolutional augmented Transformer, such as Figure 1 As shown, it includes: Step 1: Collect cloud server workload data and preprocess it, decomposing it into trend components and residual components; Step 2: Build a load prediction model, train it using trend components and residual components to obtain a trained load prediction model; use the trained load prediction model to predict the workload of the cloud server to obtain the predicted workload value. The load prediction model includes a trend information capture module, a convolutional enhanced Transformer encoder module, and a feature fusion module; The trend component is predicted by the trend information capture module, and the prediction result of the trend component is obtained. The residual components are predicted by the convolution-enhanced Transformer encoder module to obtain the prediction results of the residual components; The feature fusion module fuses the prediction results of the trend component and the prediction results of the residual component to obtain the final workload prediction value.

[0027] Example 2 The cloud service workload prediction method based on convolutional augmented Transformer described in Example 1 differs in that: Collect cloud server workload data and preprocess it, decomposing it into trend components and residual components; including: Collect cloud server workload data, including CPU utilization, memory usage, network I / O, and requests per second (QPS). The cloud server workload data is adjusted to a two-dimensional tensor X of (T, N), where T represents the length of the time series and N represents the total dimension of the features; The original input sequence X has vastly different dimensions and numerical ranges for different features (e.g., CPU utilization is between 0 and 100, and memory usage may range from hundreds to thousands of MiB), and the overall distribution of online data may drift over time. To address these issues, the input X is preprocessed first. Calculate the mean μ_i and standard deviation σ_i of each feature (each of the N features) of the two-dimensional tensor X on the time axis T, where i∈[1,N]; Each feature is normalized (using the Z-score normalization formula), as shown below: X_norm_i = (X_i - μ_i) / σ_i; Where X_norm_i represents the normalized sequence of the i-th feature, and X_i represents the two-dimensional tensor of the i-th feature; this operation transforms all features to a standard normal distribution with a mean of 0 and a variance of 1; crucially, the calculated mean vector and standard deviation vector are temporarily saved for subsequent inverse operations; after normalizing the N features, the normalized sequence X_norm is obtained; After obtaining the normalized sequence X_norm, in order to enable the model to learn different patterns in the sequence differentially, the normalized sequence X_norm is decomposed into two parts: long-term trend and short-term residuals. This is achieved through a moving average kernel, which includes setting a sliding window of size k (e.g., k=25), sliding it on the time axis of X_norm, calculating the average of all data within each sliding window, forming a new sequence, and obtaining the trend component X_tre; this component represents the macroscopic and smooth trend of the workload. By subtracting X_res = X_norm - X_tre element by element, the residual component X_res is obtained. This component is stripped of the long-term trend and mainly contains the short-term, high-frequency fluctuations of the workload and noise information. After this step, the original input X is successfully transformed into two distinct and decoupled sequences X_tre and X_res.

[0028] The trend component is predicted using the trend information capture module, and the prediction result of the trend component is obtained; including: Since the trend component X_tre changes relatively smoothly and contains low information entropy, using complex nonlinear models can easily lead to overfitting. Therefore, the trend information capture module includes a fully connected layer (Linear Layer), which feeds the trend component X_tre into the fully connected layer to learn the linear mapping relationship and outputs the prediction result X_tre_out of the trend component.

[0029] The residual components are predicted using a convolution-enhanced Transformer encoder module to obtain the prediction results for the residual components; including: The convolution-enhanced Transformer encoder module includes an embedding layer, a multi-head attention module, a convolutional module, a feedforward network module, and a decoder module; The residual component X_res maps the N-dimensional feature space to a higher-dimensional feature representation (e.g., from N=8 dimensions to d_model=512 dimensions) through an embedding layer (such as a linear layer), resulting in X_input; The output X_input of the embedding layer is simultaneously input into the multihead attention block and the convolution block for parallel processing; Among them, the multi-head attention module calculates the correlation score between any two time points to capture global dependencies; Convolutional modules extract local features from time series data while maintaining a low number of parameters. The outputs of the multi-head attention module and the convolution module are added and fused, and the result is input into the feedforward network module; The result of the summation and fusion is nonlinearly transformed by the feedforward network module. The result of the nonlinear transformation is added to the result of the summation and fusion, and then decoded by the decoder module to finally obtain the prediction result of the residual component.

[0030] The multi-head attention module includes multi-head attention operations, first residual connections, and first layer normalization. The convolution module includes convolution operations, a second residual connection, and a second normalization layer; The feedforward network module includes a position feedforward network, a third residual connection, and a third-layer normalization; First, in the multi-head attention module, the output X_input of the embedding layer undergoes multi-head attention operation, is then joined with X_input using the first residual concatenation (addition), and the calculation result is normalized using the first layer to obtain the output X_att, as shown below: X_att = LayerNorm(X_input + MultiheadAttention(X_input)); Where X_att represents the output of the multi-head attention module, LayerNorm represents the layer normalization computation (LN) of the first layer normalization, and MultiheadAttention represents the multi-head self-attention computation of the multi-head attention operation; In the convolution module, the output X_input of the embedding layer undergoes a convolution operation followed by a second residual concatenation (addition), and the calculation result is then processed through a second normalization layer to obtain the output X_conv, as shown below: X_conv = LayerNorm(X_input + Convolution(X_input)); Where X_conv represents the output of the convolution module, LayerNorm represents layer normalization, and Convolution represents the convolution operation; the convolution operation includes pointwise convolution and depthwise convolution. Pointwise convolution expands the number of input channels (to twice the original number) through one-dimensional convolution (with a kernel size of 1), and then uses the GLU activation function to reduce the number of channels back to the original number. Depthwise convolution extracts features from each channel through convolution calculation (with a kernel size of 3). During this process, appropriate padding can be used to keep the sequence length unchanged. Then, the outputs of the multi-head attention module and the convolutional module are added and fused to obtain the enhanced feature X_hid, which includes global context information and local detail information, as shown below: X_hid = X_att + X_conv; The enhanced feature X_hid is input into the feedforward network module, and the enhanced feature X_hid is nonlinearly transformed by the position-wise feed-forward network; the entire structure of "parallel processing -> fusion -> feedforward network" constitutes a complete encoder layer; The result of the nonlinear transformation is joined with the enhanced feature X_hid by a third residual connection (addition), and then the encoded output is obtained through a third-layer normalization. Finally, the encoded output is sent to the decoder module, which uses a multilayer perceptron (MLP) to decode the encoded output and finally outputs the prediction result X_res_out of the residual components (with shape (P, N)).

[0031] The feature fusion module fuses the prediction results of the trend component and the residual component to obtain the final workload prediction value; including: The prediction results X_tre_out of the trend component and X_res_out of the residual component are fused together as follows: ; in, This represents the result after feature fusion, where W is a learnable parameter (matrix). Using the mean μ_i and standard deviation σ_i from the preprocessing step Performing the reverse operation yields the final prediction result, as shown below: Y = * σ_i +μ_i; Where Y represents the final prediction result, which is the accurate prediction of N indicators, including CPU utilization, within the next P=144 minutes.

[0032] The goal of this embodiment is to perform high-precision, multi-step rolling prediction of CPU utilization for a data service API (Application Programming Interface) hosting critical business operations over a future period (e.g., the next 2-3 hours) in a Kubernetes-based cloud-native production environment. Figure 2As shown in the figure, the prediction results will serve as a key input to the downstream proactive resource scaling decision system, enabling services to be scaled up or down in advance. This effectively avoids the latency drawbacks of traditional responsive scaling strategies, ensuring service stability and resource utilization efficiency.

[0033] (1) Operating environment and data preparation The operating environment in this embodiment is a standard cloud-native technology stack; the underlying infrastructure is a Kubernetes cluster containing multiple nodes. The open-source monitoring solution Prometheus is deployed in the cluster to collect real-time performance metrics of each service (Pod) in the cluster, including CPU utilization, memory usage, network I / O, and requests per second (QPS). The collected data is stored in a time-series format.

[0034] The predictive model described in this invention is encapsulated as a standalone microservice, for example, built using the Python Flask framework and deployed in a Kubernetes cluster. The model's training and inference tasks are executed on a workstation equipped with a high-performance graphics processing unit (GPU, such as an NVIDIA RTX 4090), using the PyTorch deep learning framework.

[0035] The prediction task's data comes from Prometheus. The system sets a fixed data acquisition and prediction cycle, for example, once per minute. At each time point t, the prediction task starts, aiming to predict the workload for the next P time steps based on historical data from the past T time steps. In this embodiment, the historical review window T is set to 144 time steps, and the prediction length P is also 144 time steps, that is, using data from the past 144 minutes to predict the CPU utilization for the next 144 minutes.

[0036] (2) Steps of the prediction method (2-1) Historical data acquisition and sequence construction At the start of the prediction period, the system first sends a query request to Prometheus to obtain all relevant metrics for the target API service over the past T=144 time steps. Although the primary prediction target in this embodiment is CPU utilization, N-1 other metrics, such as memory usage and network inflow / outflow rates, are also obtained in order to build a multivariate prediction model to capture the potential correlations between the metrics.

[0037] These data are integrated into a two-dimensional tensor X of shape (T, N), where T=144, representing the length of the time series, and N represents the total dimension of the features. This tensor X is the original input sequence for this prediction task.

[0038] (2-2) Reversible instance normalization and trend decomposition The original input sequence X has vastly different dimensions and numerical ranges for different features (e.g., CPU utilization between 0 and 100, and memory usage may range from hundreds to thousands of MiB), and the overall distribution of online data may drift over time. To address these issues, the input X is first preprocessed.

[0039] Reversible instance normalization: Reference Figure 1 The "RevIN Normalization" module in the code. For the input tensor X, this module independently computes the mean μ_i and standard deviation σ_i (where i ranges from 1 to N) of each feature dimension (i.e., each of the N features) over the time axis T. Then, the Z-score normalization formula is applied to each feature X_i: X_norm_i = (X_i - μ_i) / σ_i. This operation transforms all features to a standard normal distribution with a mean of 0 and a variance of 1. Crucially, the computed mean and standard deviation vectors are temporarily saved for subsequent inverse operations. The output of this step is the normalized sequence X_norm.

[0040] Trend decomposition: After obtaining the normalized sequence X_norm, in order to enable the model to learn different patterns in the sequence in a differentiated manner, it is decomposed into two parts: long-term trend and short-term residual.

[0041] After this step, the original input X is successfully transformed into two distinct and decoupled sequences X_tre and X_res.

[0042] (2-3) Parallelized Dual-Path Prediction One of the core innovations of this invention lies in using different models to process the two decomposed components in order to achieve optimal efficiency and accuracy.

[0043] Trend component prediction: See attached document Figure 1The "Linear" path is used. Since the trend component X_tre changes relatively smoothly and contains low information entropy, using complex nonlinear models can easily lead to overfitting. Therefore, a simple fully connected linear layer is used to process it. The X_tre sequence is fed into this linear layer, which learns a linear mapping relationship and directly outputs the trend prediction result X_tre_out for the next P=144 time steps.

[0044] Residual component prediction: See attached document Figure 1 The convolution-enhanced Transformer encoder part is used in this section. The residual component X_res contains a large amount of nonlinear, highly dynamic details, which is the challenge of the prediction task. A convolution-enhanced Transformer encoder specifically designed for this purpose is used for processing.

[0045] To obtain the final complete prediction, the prediction results are weighted and summed: = W * X_tre_out + (1-W) * X_res_out. Here, W is a learnable parameter or matrix that is automatically optimized during model training to determine the optimal weighting of the trend and residual components in the final prediction. It is the combined prediction result in the normalized space.

[0046] The final and crucial step is to restore the prediction results to the true physical scale. This involves retrieving the saved mean vector μ_i and standard deviation vector σ_i, and then... Perform the reverse operation, and the final output Y tensor is the accurate prediction of N indicators, including CPU utilization, memory usage, network I / O, and requests per second (QPS), within the next P=144 minutes.

[0047] Example 3 A computer device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the cloud service workload prediction method based on convolutional augmented Transformer as described in Embodiment 1 or 2.

[0048] Example 4 A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the cloud service workload prediction method based on convolutional augmented Transformer as described in Embodiment 1 or 2.

[0049] Example 5 A cloud service workload prediction system based on convolutional augmented Transformer includes: The preprocessing module is configured to: collect cloud server workload data and preprocess it, decomposing it into trend components and residual components; The model prediction module is configured to: build a load prediction model, train it using trend components and residual components to obtain a trained load prediction model; and use the trained load prediction model to predict the workload of cloud servers to obtain the predicted workload value. The load prediction model includes a trend information capture module, a convolutional enhanced Transformer encoder module, and a feature fusion module; The trend component is predicted by the trend information capture module, and the prediction result of the trend component is obtained. The residual components are predicted by the convolution-enhanced Transformer encoder module to obtain the prediction results of the residual components; The feature fusion module fuses the prediction results of the trend component and the prediction results of the residual component to obtain the final workload prediction value.

Claims

1. A cloud service workload prediction method based on convolutional augmented Transformer, characterized in that, include: Step 1: Collect cloud server workload data and preprocess it, decomposing it into trend components and residual components; Step 2: Build a load prediction model, train it using trend components and residual components to obtain a trained load prediction model; use the trained load prediction model to predict the workload of the cloud server to obtain the predicted workload value. The load prediction model includes a trend information capture module, a convolutional enhanced Transformer encoder module, and a feature fusion module; The trend component is predicted by the trend information capture module, and the prediction result of the trend component is obtained. The residual components are predicted by the convolution-enhanced Transformer encoder module to obtain the prediction results of the residual components; The feature fusion module fuses the prediction results of the trend component and the prediction results of the residual component to obtain the final workload prediction value.

2. The cloud service workload prediction method based on convolutional augmented Transformer according to claim 1, characterized in that, Collect cloud server workload data and preprocess it, decomposing it into trend components and residual components; including: Collect cloud server workload data, including CPU utilization, memory usage, network I / O, and requests per second (QPS). The cloud server workload data is adjusted to a two-dimensional tensor X of (T, N), where T represents the length of the time series and N represents the total dimension of the features; Calculate the mean μ_i and standard deviation σ_i of each feature of the two-dimensional tensor X on the time axis T, where i∈[1,N]; Normalize each feature as follows: X_norm_i = (X_i - μ_i) / σ_i; Where X_norm_i represents the normalized sequence of the i-th feature, and X_i represents the two-dimensional tensor of the i-th feature; the normalized sequence X_norm is obtained by normalizing the N features; The normalized sequence X_norm is decomposed into two parts: long-term trend and short-term residuals. This includes setting a sliding window of size k and sliding it on the time axis of X_norm, calculating the average of all data within each sliding window, forming a new sequence, and obtaining the trend component X_tre. The residual component X_res is obtained by subtracting X_res element by element, X_res = X_norm - X_tre.

3. The cloud service workload prediction method based on convolutional augmented Transformer according to claim 2, characterized in that, The trend component is predicted using the trend information capture module, and the prediction result of the trend component is obtained; including: The trend information capture module includes a fully connected layer. The trend component X_tre is fed into the fully connected layer to learn the linear mapping relationship and output the prediction result X_tre_out of the trend component.

4. The cloud service workload prediction method based on convolutional augmented Transformer according to claim 3, characterized in that, The residual components are predicted using a convolution-enhanced Transformer encoder module to obtain the prediction results for the residual components; including: The convolution-enhanced Transformer encoder module includes an embedding layer, a multi-head attention module, a convolutional module, a feedforward network module, and a decoder module; The residual component X_res is mapped from the N-dimensional feature space to a higher-dimensional feature representation through the embedding layer, resulting in X_input; The output X_input of the embedding layer is simultaneously input into the multi-head attention module and the convolution module for parallel processing. Among them, the multi-head attention module calculates the correlation score between any two time points to capture global dependencies; Convolutional modules extract local features from time series data; The outputs of the multi-head attention module and the convolution module are added and fused, and the result is input into the feedforward network module; The result of the summation and fusion is nonlinearly transformed by the feedforward network module. The result of the nonlinear transformation is added to the result of the summation and fusion, and then decoded by the decoder module to finally obtain the prediction result of the residual component.

5. The cloud service workload prediction method based on convolutional augmented Transformer according to claim 4, characterized in that, The multi-head attention module includes multi-head attention operations, a first residual connection, and a first-layer normalization. The convolution module includes convolution operations, a second residual connection, and a second normalization layer; The feedforward network module includes a position feedforward network, a third residual connection, and a third-layer normalization; First, in the multi-head attention module, the output X_input of the embedding layer undergoes multi-head attention operation, is then connected to X_input via a first residual, and the calculation result is normalized in the first layer to obtain the output X_att, as shown below: X_att = LayerNorm(X_input + MultiheadAttention(X_input)); Where X_att represents the output of the multi-head attention module, LayerNorm represents the layer normalization calculation of the first normalization layer, and MultiheadAttention represents the multi-head self-attention calculation of the multi-head attention operation; In the convolution module, the output X_input of the embedding layer undergoes a convolution operation followed by a second residual connection. The computation result is then processed through a second-layer normalization to obtain the output X_conv, as shown below: X_conv = LayerNorm(X_input + Convolution(X_input)); Where X_conv represents the output of the convolution module, LayerNorm represents layer normalization, and Convolution represents the convolution operation; the convolution operation includes point convolution and depthwise convolution. Point convolution expands the number of input channels through one-dimensional convolution and uses the GLU activation function to reduce the number of channels back to the original number; depthwise convolution extracts features from each channel through convolution calculation. Then, the outputs of the multi-head attention module and the convolutional module are added and fused to obtain the enhanced feature X_hid, which includes global context information and local detail information, as shown below: X_hid = X_att + X_conv; The enhanced feature X_hid is input into the feedforward network module, and the enhanced feature X_hid is nonlinearly transformed through the position feedforward network. The result after nonlinear transformation is connected to the enhanced feature X_hid through a third residual connection, and then the encoded output is obtained through a third layer of normalization; Finally, the encoded output is sent to the decoder module, which uses a multilayer perceptron to decode the encoded output and finally outputs the prediction result X_res_out of the residual components.

6. The cloud service workload prediction method based on convolutional augmented Transformer according to claim 5, characterized in that, The feature fusion module fuses the prediction results of the trend component and the residual component to obtain the final workload prediction value; including: The prediction results X_tre_out of the trend component and X_res_out of the residual component are fused together as follows: ； in, This represents the result after feature fusion, where W is a learnable parameter; Using the mean μ_i and standard deviation σ_i from the preprocessing step Performing the reverse operation yields the final prediction result, as shown below: Y = * σ_i +μ_i; Where Y represents the final prediction result.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, The processor executes the computer program to implement the steps of a cloud service workload prediction method based on convolutional augmented Transformer.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of a cloud service workload prediction method based on convolutional augmented Transformer.

9. A cloud service workload prediction system based on convolutional augmented Transformer, characterized in that, include: The preprocessing module is configured to: collect cloud server workload data and preprocess it, decomposing it into trend components and residual components; The model prediction module is configured to: build a load prediction model, train it using trend components and residual components to obtain a trained load prediction model; and use the trained load prediction model to predict the workload of cloud servers to obtain the predicted workload value. The load prediction model includes a trend information capture module, a convolutional enhanced Transformer encoder module, and a feature fusion module; The trend component is predicted by the trend information capture module, and the prediction result of the trend component is obtained. The residual components are predicted by the convolution-enhanced Transformer encoder module to obtain the prediction results of the residual components; The feature fusion module fuses the prediction results of the trend component and the prediction results of the residual component to obtain the final workload prediction value.

Images