An online integrated load prediction method for cloud service cold start

Abstract

This invention relates to an online integrated load prediction method for cold starts of cloud services, comprising the following steps: obtaining a time series of cloud service workloads as input; performing multi-resolution slicing and channel-independent dimension alignment operations on the time series using a multi-input multi-output subnet to obtain multiple sets of slices at different resolutions; inputting each set of slices into a corresponding online Transformer model and training it online; and integrating the outputs of multiple online Transformer models online to obtain the final load prediction result. Compared with existing technologies, this invention has advantages such as improved prediction accuracy, computational efficiency, and strong adaptability.

Description

Technical Field

[0001] This invention relates to the field of cloud service technology, and in particular to an online integrated load prediction method for cold start of cloud services. Background Technology

[0002] In the field of cloud computing load forecasting, early statistical methods (such as ARIMA and ETS) and some machine learning methods (such as decision trees and support vector machines) have been widely used. These methods are effective in handling relatively stable load patterns, but their limitations become increasingly apparent when faced with the complex and ever-changing workloads in cloud computing environments. Cloud computing workloads typically exhibit diverse and dynamic characteristics; even if periodic patterns exist, their peak amplitudes fluctuate over time. Traditional methods struggle to handle concept drift problems online, meaning they cannot adapt to changes in workload characteristics in a timely manner. Regarding long-term forecasting, due to their relatively simple model structures, forecast accuracy drops significantly as the forecast time span increases, failing to meet the demands of cloud computing systems for accurate forecasting over long periods. Furthermore, traditional methods are almost powerless against the cold start problem faced by newly introduced cloud services (such as frequently deployed new functions in FaaS models), i.e., load forecasting without historical data, because they heavily rely on large amounts of historical data to train their models.

[0003] In recent years, Transformer-based neural network models have received widespread attention in the field of time series forecasting. These models have demonstrated good performance in long-term forecasting tasks and also have certain advantages in learning transfer representations. However, almost all of these studies focus primarily on batch learning settings, which cannot adapt to the rapid changes in workloads in cloud computing environments in real time. Due to their high computational requirements, when applied to load forecasting in practice, they can only be retrained at long intervals (such as daily), making it impossible for them to capture real-time changes in workloads within the same day. For example, in a cloud service system with high real-time requirements, the workload may change significantly multiple times within a day due to various factors (such as sudden changes in user behavior, the impact of external events, etc.), but Transformer-based models cannot respond to these changes between two retraining sessions, resulting in a disconnect between the predicted results and the actual situation. Although some studies have attempted to enable deep neural networks to update online by improving the update structure (such as using Temporal Convolutional Networks as the framework network), these methods still fail to fully address all the challenges of cloud computing load forecasting, especially the cold start problem, i.e., when faced with new cloud service tasks and limited historical data, the model's predictive performance remains unsatisfactory. Summary of the Invention

[0004] The purpose of this invention is to provide an online integrated load forecasting method for cold starts of cloud services, specifically designed for online, migration, and long-term time series forecasting, and particularly suitable for cold start tasks in cloud services.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] An online integrated load forecasting method for cold start of cloud services includes the following steps:

[0007] Obtain the time series of cloud service workloads as input;

[0008] The time series is segmented into multiple resolution slices and channel-independent dimension alignment is performed using a multiple-input multiple-output subnet to obtain multiple sets of slices with different resolutions.

[0009] Each set of slices is input into the corresponding online Transformer model and trained online.

[0010] The outputs of multiple online Transformer models are integrated online to obtain the final load prediction result.

[0011] The multi-resolution slice segmentation specifically refers to:

[0012] For the input time series X t ∈R L×M Where L is the backtracking window length and M is the number of data channels, according to the formula Divide the time series into slices X of different sizes. P ∈R M×P×N Where P is the slice size and N is the number of slices; before segmentation, the original sequence is padded with P repeated end values ​​to ensure the integrity of the segmentation.

[0013] The channel-independent dimension alignment operation is specifically as follows:

[0014] Perform RevIn normalization and linear embedding operations on each group of slices to align their dimensions on the last axis. Among them, P i Let be the size of the i-th slice.

[0015] The online Transformer model performs the following operations on input slices of different resolutions:

[0016] The slices processed by the multi-input multi-output subnet are subjected to input embedding operation, which maps them to vector space and adds position encoding to introduce the position information of elements in the sequence;

[0017] The slices, after input embedding and position encoding, are fed into an attention block containing multiple attention layers. The output of the last attention layer is flattened and then mapped through a linear layer to obtain the final load prediction result f(X)∈R. H Where H represents the prediction time span, the attention layer includes multi-head self-attention, skip connections, batch normalization, and a feedforward network. The specific calculation process of the attention layer is as follows: the input of the attention layer is added to the output of MHSA through skip connections, then batch normalization is performed, the feedforward network processes the normalized result, and finally, batch normalization and skip connection operations are performed again to obtain the output of the attention layer, which serves as the input of the next attention layer. The multi-head self-attention output is MHSA(X). P = Concat(head1,…,head) h W O W O ∈R hD′×D h is the number of heads, D is the hidden dimension, D′ is the dimension of D after mapping, and W O To output the weight matrix, Concat refers to the operation of concatenating multiple tensors along a specific dimension. i This represents the output of the i-th head in the multi-head self-attention mechanism.

[0018] Based on channel independence, the online Transformer model generates prediction results for all channels simultaneously, i.e., f(X)∈R. M×H Where M is the number of channels.

[0019] During training, the online Transformer model updates the weights within the attention mechanism based on the EMA gradient.

[0020] Calculate the EMA gradient: Update the EMA gradient according to the following formula: in, The gradient of the attention layer over time. Here, γ is the EMA gradient, and γ is the smoothing coefficient.

[0021] Generate fitness coefficients: update the obtained EMA gradients As input, fitness coefficients α and β are generated through a linear layer: Where Flatten represents tiling operation and Linear represents linear operation;

[0022] Weight update: The weight matrix W of the multi-head attention mechanism is adapted by weight adjustment, calculated as follows: Where ⊙ represents element-wise multiplication;

[0023] Feature adaptation: The embedding C of the attention mechanism is updated based on the weight matrix after weight update, and then feature adaptation is performed. The calculation method is as follows:

[0024] The online integration selects either an online scaling model or a perturbation leader strategy based on the requirements for prediction efficiency and accuracy.

[0025] The online scaling model uses EGD as the ensemble strategy, and the decision space is a d-dimensional simplex Δ={w t |w t,i ≥0 and ∥w t ∥1=1}, where t is the time step, w t,i Let x be the weight of the i-th Transformer model at step t, given an online data sequence x, a prediction target Y, and d Transformer models with different parameters but some parameters shared. The goal of EGD is to minimize the prediction error, and the update rule is:

[0026]

[0027] in, η is the normalization factor, and η is the learning rate.

[0028] The online scaling model consists of an input embedding layer, a multi-head self-attention layer equipped with an online adapter, and an output linear layer. The input embedding layer stacks the predicted value F and the predicted target Y together, and the linear layer maps the time dimension H to the hidden dimension D. After multi-head self-attention calculation, the output result is flattened and then input into the linear layer to generate scaling weights s = SoftMax(Linear(Flatten(MHSA(F))) + w). Based on the scaling weights, the online scaling model outputs optimized prediction results. In this context, MHSA represents multi-head self-attention computation, Flatten represents tiling operation, Linear represents linear operation, SoftMax represents normalization operation, and w represents weight parameters. By applying additional weighted adjustments to the feature information extracted through the multi-head self-attention mechanism, the model can more flexibly emphasize or weaken the influence of certain features based on the value of w when generating scaling weights s.

[0029] The decision space for the follow-the-disturbance-leader strategy is Π = π. t |π t,i ∈0,1 and |π t |1=1,π t,i This refers to the value of the i-th strategy or choice at time step t, |π t |1 represents the vector π tThe L1 norm, in an online setting, given input X and objective Y, the follow-perturbation leader strategy aims to minimize cumulative loss. Its core principle is to explore the decision space by introducing random perturbations, and the update mechanism is based on... The implementation is achieved, where σ(π) is the perturbation added to the cumulative loss, and t is the time step.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] (I) Significantly improve prediction accuracy

[0032] In online prediction tasks, this invention can more accurately capture the changing trends of cloud service load, improve prediction accuracy, provide a more reliable basis for cloud system resource allocation, and thus improve the overall system performance.

[0033] In transfer learning prediction tasks, this invention can quickly adapt to new tasks and data environments, effectively overcome the challenges faced by traditional models in transfer learning, and greatly improve the model's versatility and practicality.

[0034] (II) High-efficiency computing performance

[0035] This invention ensures high-accuracy prediction while maintaining low computational overhead. In practical cloud service scenarios, this means that the invention can respond more quickly to real-time load changes, provide accurate prediction results in a timely manner, effectively meet the stringent efficiency requirements of cloud computing systems, and improve system operating efficiency.

[0036] The online learning capability of the model in this invention further enhances computational efficiency. Through online adaptation mechanisms (such as gradient smoothing in the online Transformer framework and dynamic weight adjustment in the online ensemble module), this invention can update the model in real time using newly incoming data, avoiding the huge computational costs caused by frequent retraining. In practical applications, compared with traditional models that are periodically retrained, this invention can save a significant amount of computational resources while maintaining the model's sensitivity to load changes, ensuring the timeliness and accuracy of predictions.

[0037] (III) Strong adaptability

[0038] The unique online adapter design enables this invention to adapt in real-time to concept drift phenomena in cloud service load data. In a cloud computing environment, workload characteristics can change at any time due to various factors (such as changes in user behavior, the launch of new applications, etc.). This invention uses an online adapter to dynamically adjust model parameters, quickly adapting to these changes. For example, in long-term monitoring of actual cloud service load data, when encountering sudden traffic peaks or troughs, this invention can quickly adjust the prediction model, while traditional methods often require a longer readjustment period, leading to a significant increase in prediction errors during those periods. This feature allows this invention to consistently provide predictions that closely match actual load conditions, providing effective support for cloud system resource management.

[0039] This invention demonstrates superior capabilities in addressing the cold start problem. In cloud services, new functions or tasks are constantly being deployed, often facing the challenge of insufficient historical data for training accurate prediction models. This invention employs a multi-resolution slice learning and ensemble strategy to extract valuable information from limited historical data and leverages transfer learning from knowledge gained on other relevant tasks or datasets. The prediction accuracy is significantly higher than traditional methods and some neural network methods, providing strong support for efficient resource allocation and rapid startup in cloud services. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0041] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0042] This embodiment provides an online integrated load prediction method for cold start of cloud services, including the following steps:

[0043] S1, obtain the time series of cloud service workloads as input;

[0044] S2, using a multi-input multi-output subnet to perform multi-resolution slicing and channel-independent dimension alignment operations on the time series to obtain multiple sets of slices with different resolutions;

[0045] S3, input each group of slices into the corresponding online Transformer model and train it online;

[0046] S4 integrates the outputs of multiple online Transformer models online to obtain the final load prediction result.

[0047] like Figure 1As shown, the above method mainly consists of three core parts: a multiple-input multiple-output (MIMO) subnet, an online Transformer model (Online Transformer Backbone), and an online ensemble module. These parts work together to achieve efficient and accurate load forecasting.

[0048] (1) Multiple-input multiple-output (MIMO) subnet

[0049] The input time series first enters the MIMO subnet, which divides it into slices of different resolutions (e.g., by different size values). After each channel is processed independently, RevIn normalization and linear embedding operations are performed. The processed multiple sets of slices are then input into the online Transformer model.

[0050] For the input time series X t ∈R L×M Where L is the backtracking window length and M is the number of data channels, a multi-resolution slice segmentation strategy is adopted to enhance the model's ability to learn features at different scales. According to the formula... Divide the time series into slices X of different sizes. P ∈R M×P×N Where P is the slice size and N is the number of slices; before segmentation, the original sequence is padded with P repeated last values ​​to ensure the integrity of the segmentation. For example, for a time series of length 100 (L=100) and number of channels 5 (M=5), when a slice size of 10 is selected, the number of slices can be calculated according to the formula. This divides the original time series into 10 slices of size 5×10.

[0051] Different slice sizes are used to capture different features in the sequence. Larger slice sizes (e.g., P=128) help capture high-frequency features and improve computational efficiency by reducing the computational cost of the attention mechanism (the computational complexity of the attention mechanism is related to the square of the sequence length; larger slices shorten the effective sequence length). Smaller slice sizes (e.g., P=16) maintain a larger receptive field, which is beneficial for capturing low-frequency features. For example, when analyzing cloud service load data, high-frequency features may reflect rapid fluctuations in load in the short term (e.g., sudden request spikes), while low-frequency features may reflect long-term load trends (e.g., daily or weekly load cycles).

[0052] In this embodiment, channels are set independently, meaning that the processing of each channel is independent. When processing the slices, RevIn normalization and linear embedding operations are performed on the slices of each group to align their dimensions on the last axis. Among them, P i Let be the size of the i-th slice. After this processing, multiple sets of slices at different resolutions will be input into a shared Transformer-based prediction model for further processing.

[0053] (2) Online Transformer Model

[0054] The online Transformer model receives input from the MIMO subnet. After input embedding and position encoding, it is processed through an attention block containing a multi-head self-attention (MHSA) mechanism. Within the attention block, MHSA calculation, skip connections, batch normalization, and feedforward network (FFN) operations are performed sequentially. The output of the last attention block is flattened and processed by a linear layer to obtain the preliminary prediction result. At the same time, during training, the gradient is processed using exponential moving average (EMA) to adapt to changes in online data, and its output is used as the input of the online ensemble module.

[0055] Specifically, the online Transformer model performs the following operations on slices of input at different resolutions:

[0056] 1. After processing the slices through the multi-input multi-output subnet, an input embedding operation is performed to map them to a suitable vector space and add positional encoding to incorporate the positional information of elements in the sequence. This step enables the model to distinguish input elements at different positions, which is crucial for time series prediction because the order of data in a time series contains important information.

[0057] 2. The slices after input embedding and position encoding are fed into an attention block containing multiple attention layers. The output of the last attention layer is flattened and then mapped through a linear layer to obtain the final load prediction result f(X)∈R. H , where H is the prediction time span.

[0058] The attention layer comprises multi-head self-attention, skip connections, batch normalization, and a feedforward network (FFN). The specific computation process of the attention layer is as follows: the input of the attention layer is added to the output of MHSA via skip connections, then batch normalization is performed. The feedforward network processes the normalized result, and finally, batch normalization and skip connections are performed again to obtain the output of the attention layer, which serves as the input to the next attention layer. The multi-head self-attention (MHSA) mechanism is the core. MHSA allows the model to simultaneously focus on information from different representation subspaces and different locations. The output of multi-head self-attention is MHSA(X). P= Concat(head1,…,head) h W O W O ∈R hD′×D h is the number of heads, D is the hidden dimension, D′ is the dimension of D after mapping, and W O To output the weight matrix, Concat refers to the operation of concatenating multiple tensors along a specific dimension. i This represents the output of the i-th head in the multi-head self-attention mechanism.

[0059] Based on channel independence, the online Transformer model generates prediction results for all channels simultaneously, i.e., f(X)∈R. M×H .

[0060] During online Transformer model training, updating the weights within the attention mechanism based on the EMA (Exponential Moving Average) gradient is a crucial step in improving the model's adaptability to time-series data. When processing time-series data, noise and non-stationarity can cause gradient fluctuations, which the EMA gradient can effectively smooth out, allowing the model to better capture the temporal information in the data. The detailed weight update process is as follows:

[0061] 1. Calculate the EMA gradient: In online training, the concept of EMA is introduced to mitigate the impact of time-series data noise on the gradient of individual samples. The EMA gradient is updated according to the following formula: in, The gradient of the attention layer over time. Here, γ represents the EMA gradient, and γ is the smoothing coefficient. In this way, the model can, to some extent, ignore short-term gradient fluctuations and focus on long-term trends.

[0062] 2. Generate fitness coefficients: Update the obtained EMA gradients. As input, fitness coefficients α and β are generated through a linear layer: Here, Flatten represents the tiling operation, and Linear represents the linear operation. The fitness coefficients here include the weight fitness coefficient α and the feature fitness coefficient β, which will be used for subsequent weight updates and feature adaptations, respectively.

[0063] 3. Weight Update: The weight matrix W of the multi-head attention mechanism is adapted by weight adjustment, calculated as follows: Where ⊙ represents element-wise multiplication.

[0064] This embodiment uses query embedding as an example for explanation. First, the weight matrix W of the query embedding is... QWeight adaptation is performed, and the calculation method is as follows: Where ⊙ denotes element-wise multiplication. Weights after weight adaptation. It will be used in subsequent multi-head self-attention (MHSA) calculations, replacing the original weight matrix W. Q In this way, the model can dynamically adjust the weights within the attention mechanism based on the EMA gradient, making it better adaptable to constantly changing data distributions.

[0065] 4. Feature Adaptation: The embedding C of the attention mechanism is updated based on the weight matrix after weight updates, and then feature adaptation is performed. The calculation method is as follows:

[0066] In the example of query embedding, the weight-adjusted query embedding Then feature adaptation is performed, calculated as follows: The feature-adapted query embedding will play a role in multi-head self-attention computation.

[0067] Through the above steps, the model can effectively update the weights within the attention mechanism based on the EMA gradient, thereby better adapting to changes in time series data and improving the model's predictive performance. This dynamic update mechanism is significant in dealing with issues such as concept drift, long-term prediction, and cold start, enabling the model to maintain high accuracy and adaptability in constantly changing environments.

[0068] (3) Online integration module

[0069] The online ensemble module includes two implementations: Online Scaling and Follow the Perturbed Leader (FTPL). Online Scaling integrates the predictions from multiple subnets using an exponential gradient descent (EGD) strategy and an optimized activation function. FTPL, on the other hand, introduces random perturbations to explore the decision space to select or combine subnet predictions, ultimately outputting the model's predicted values.

[0070] (31) Online Scaling

[0071] The online scaling model employs EGD (Exponential Gradient Descent) as the ensemble strategy, and the decision space is a d-dimensional simplex Δ={w t |w t,i ≥0 and ∥w t ∥1=1}, where t is the time step, w t,i Let x be the weight of the i-th Transformer model at step t. Given an online data sequence x, a prediction target Y, and d Transformer models with different parameters but some parameters shared. The goal of EGD is to minimize the prediction error, and the update rule is:

[0072]

[0073] in, η is the normalization factor, and η is the learning rate.

[0074] Theoretical analysis proves that the EGD strategy can achieve a sublinear regret bound, i.e.:

[0075]

[0076] Where T is the total time step; L is the loss function; w t This represents the weight vector selected using the Exponential Gradient Descent (EGD) strategy at time step 1; u is a general variable representing any vector in the entire time series prediction process from all possible weight vector selection spaces; O is Big Onotation, used to describe the asymptotic growth rate or upper bound of the complexity of the function; inf u The infimum operation is used to find the minimum value among all possible weight vector choices. The loss value corresponding to the weight vector that reaches the minimum value.

[0077] To address the issue of lag in the response of the EGD method when faced with significant changes in data distribution (i.e., the "slow switching phenomenon"), an improved activation function based on the recent expert loss is introduced. This activation function aims to quickly adjust the weight allocation of different experts based on their recent prediction performance (measured by the loss function), enabling the model to adapt to changes in data distribution more promptly, reducing the impact of the "slow switching phenomenon," and improving the accuracy and adaptability of predictions.

[0078] The online scaling model consists of an input embedding layer, a multi-head self-attention layer equipped with an online adapter, and an output linear layer. The input embedding layer stacks the predicted value F and the predicted target Y together, and the linear layer maps the time dimension H to the hidden dimension D. After multi-head self-attention computation, the output is flattened and then input into the linear layer to generate scaling weights s = SoftMax(Linear(Flatten(MHSA(F))) + w). Based on the scaling weights, the online scaling model outputs optimized prediction results. In this context, MHSA represents multi-head self-attention computation, Flatten represents tiling operation, Linear represents linear operation, SoftMax represents normalization operation, and w represents weight parameters. By applying additional weighted adjustments to the feature information extracted through the multi-head self-attention mechanism, the model can more flexibly emphasize or weaken the influence of certain features based on the value of w when generating scaling weights s.

[0079] (2) Follow the Perturbed Leader (FTPL)

[0080] FTPL is another introduced online ensemble method, whose decision space is Π=π. t |π t,i ∈0,1 and |π t |1=1,π t,i This refers to the value of the i-th strategy or choice at time step t, |π t |1 represents the vector π t The L1 norm. In the online setting, given input X and objective Y, the follow-perturbation leader strategy aims to minimize cumulative loss. Its core principle is to explore the decision space by introducing random perturbations, and the update mechanism is based on... The implementation is given by σ(π), which is a perturbation added to the cumulative loss, typically from a predefined noise distribution, such as a uniform distribution, and t is the time step.

[0081] Theoretical analysis proves that, under certain conditions, the expected regret bound of FTPL is:

[0082]

[0083] Where c is a coefficient or weight associated with the loss function L, which is used to adjust the importance or scale of the loss function when calculating the regret bound. L represents the loss function, which measures the difference or error between the model's prediction and the true value. E is the expectation.

[0084] FTPL offers the advantage of performing well in both convex and non-convex optimization settings, and due to its non-parametric nature, it boasts higher efficiency compared to EGD. In practical applications, FTPL-based models excel in terms of parameter count and inference speed, enabling more flexible deployment in resource-constrained environments while maintaining high predictive performance. Although its performance is slightly inferior to OS-based models in most comparisons, it holds significant application value in specific scenarios (such as real-time systems with high resource and speed requirements).

[0085] In online prediction tasks, the algorithm proposed in this invention demonstrates superior performance. It achieves significant progress on multiple datasets compared to previous state-of-the-art methods (such as OneNet). For example, when predicting the FaaS dataset (covering FaaS_Small, FaaS_Medium, and FaaS_Large), the mean absolute error (MAE) of this invention is reduced by approximately 11.7% compared to OneNet. When processing public datasets (such as ETTh1, ETTh2, etc.), the mean squared error (MSE) of this invention is reduced by an average of 13.9% compared to traditional statistical models (such as STL, SARIMA, etc.) and other deep learning models (such as DLinear, TimesNet, etc.). This means that this invention can more accurately capture the changing trends of cloud service load, providing a more reliable basis for cloud system resource allocation, thereby improving the overall system performance.

[0086] In transfer learning prediction tasks, when the model is transferred from one dataset to another (such as from FaaS to IaaS or between datasets of different resolutions), this invention maintains high prediction accuracy even with unknown data distributions. Taking the FaaS→IaaS transfer task as an example, the MSE of this invention is reduced by approximately 15.3%, the MAE by approximately 14.1%, and the weighted average absolute percentage error (WMAPE) by approximately 26.3% compared to OneNet. This fully demonstrates that this invention can quickly adapt to new tasks and data environments, effectively overcome the challenges faced by traditional models in transfer learning, and greatly improve the model's versatility and practicality.

[0087] This invention maintains low computational overhead while ensuring high-accuracy predictions. Compared to other ensemble models (such as OneNet), its number of parameters is significantly reduced. For example, the number of parameters in this invention based on FTPL is only 16.7% of that in OneNet, which greatly reduces the model's storage and computing resource requirements. In terms of inference speed, this invention based on FTPL performs exceptionally well, with inference time reduced by approximately 39.6% compared to OneNet, while throughput exceeds OneNet by more than 800 times. In real-world cloud service scenarios, this means that this invention can respond more quickly to real-time load changes, provide accurate prediction results in a timely manner, effectively meet the stringent efficiency requirements of cloud computing systems, and improve system operating efficiency.

[0088] Furthermore, in the load prediction experiment of the newly deployed function on the FaaS platform, the present invention can quickly establish an effective prediction model with only a small amount of initial data. Its prediction accuracy is significantly higher than that of traditional methods and some neural network methods, providing a strong guarantee for efficient resource allocation and rapid startup of cloud services.

[0089] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. An online integrated load forecasting method for cold start of cloud services, characterized in that, Includes the following steps: Obtain the time series of cloud service workloads as input; The time series is segmented into multiple resolution slices and channel-independent dimension alignment is performed using a multiple-input multiple-output subnet to obtain multiple sets of slices with different resolutions. Each set of slices is input into the corresponding online Transformer model and trained online. The outputs of multiple online Transformer models are integrated online to obtain the final load prediction result.

2. The online integrated load forecasting method for cold start of cloud services according to claim 1, characterized in that, The multi-resolution slice segmentation specifically refers to: For the input time series X t ∈R L×M Where L is the backtracking window length and M is the number of data channels, according to the formula Divide the time series into slices X of different sizes. P ∈R M×P×N Where P is the slice size and N is the number of slices; before segmentation, the original sequence is padded with P repeated end values ​​to ensure the integrity of the segmentation.

3. The online integrated load forecasting method for cold start of cloud services according to claim 2, characterized in that, The channel-independent dimension alignment operation is specifically as follows: Perform RevIn normalization and linear embedding operations on each group of slices to align their dimensions on the last axis. Among them, P i Let be the size of the i-th slice.

4. The online integrated load forecasting method for cold start of cloud services according to claim 1, characterized in that, The online Transformer model performs the following operations on input slices of different resolutions: The slices processed by the multi-input multi-output subnet are subjected to input embedding operation, which maps them to vector space and adds position encoding to introduce the position information of elements in the sequence; The slices, after input embedding and position encoding, are fed into an attention block containing multiple attention layers. The output of the last attention layer is flattened and then mapped through a linear layer to obtain the final load prediction result f(X)∈R. H Where H represents the prediction time span, the attention layer includes multi-head self-attention, skip connections, batch normalization, and a feedforward network. The specific calculation process of the attention layer is as follows: the input of the attention layer is added to the output of MHSA through skip connections, then batch normalization is performed, the feedforward network processes the normalized result, and finally, batch normalization and skip connection operations are performed again to obtain the output of the attention layer, which serves as the input of the next attention layer. The multi-head self-attention output is MHSA(X). P = Concat(head1,…,head) h W O W O ∈R hD′×D h is the number of heads, D is the hidden dimension, D′ is the dimension of D after mapping, and W O To output the weight matrix, Concat refers to the operation of concatenating multiple tensors along a specific dimension. i This represents the output of the i-th head in the multi-head self-attention mechanism.

5. The online integrated load forecasting method for cold start of cloud services according to claim 4, characterized in that, Based on channel independence, the online Transformer model generates prediction results for all channels simultaneously, i.e., f(X)∈R. M×H Where M is the number of channels.

6. The online integrated load forecasting method for cold start of cloud services according to claim 1, characterized in that, During training, the online Transformer model updates the weights within the attention mechanism based on the EMA gradient. Calculate the EMA gradient: Update the EMA gradient according to the following formula: in, The gradient of the attention layer over time. Here, γ is the EMA gradient, and γ is the smoothing coefficient. Generate fitness coefficients: update the obtained EMA gradients As input, fitness coefficients α and β are generated through a linear layer: Where Flatten represents tiling operation and Linear represents linear operation; Weight update: The weight matrix W of the multi-head attention mechanism is adapted by weight adjustment, calculated as follows: Where ⊙ represents element-wise multiplication; Feature adaptation: The embedding C of the attention mechanism is updated based on the weight matrix after weight update, and then feature adaptation is performed. The calculation method is as follows:

7. The online integrated load forecasting method for cold start of cloud services according to claim 1, characterized in that, The online integration selects either an online scaling model or a perturbation leader strategy based on the requirements for prediction efficiency and accuracy.

8. The online integrated load forecasting method for cold start of cloud services according to claim 7, characterized in that, The online scaling model uses EGD as the ensemble strategy, and the decision space is a d-dimensional simplex Δ={w t |w t,i ≥0 and ∥w t ∥1=1}, where t is the time step, w t,i Let x be the weight of the i-th Transformer model at step t, given an online data sequence x, a prediction target Y, and d Transformer models with different parameters but some parameters shared. The goal of EGD is to minimize the prediction error, and the update rule is: in, η is the normalization factor, and η is the learning rate.

9. The online integrated load forecasting method for cold start of cloud services according to claim 8, characterized in that, The online scaling model consists of an input embedding layer, a multi-head self-attention layer equipped with an online adapter, and an output linear layer. The input embedding layer stacks the predicted value F and the predicted target Y together, and the linear layer maps the time dimension H to the hidden dimension D. After multi-head self-attention calculation, the output result is flattened and then input into the linear layer to generate scaling weights s = SoftMax(Kinear(Flatten(MHSA(F))) + w). Based on the scaling weights, the online scaling model outputs optimized prediction results. Where MHSA represents multi-head self-attention computation, Flatten represents tiling operation, Linear represents linear operation, SoftMax represents normalization operation, and w represents weight parameter.

10. The online integrated load forecasting method for cold start of cloud services according to claim 7, characterized in that, The decision space for the follow-the-disturbance-leader strategy is Π = π. t |π t,i ∈0,1 and |π t |1=1,π t,i This refers to the value of the i-th strategy or choice at time step t, |π t |1 represents the vector π t The L1 norm, in an online setting, given input X and objective Y, the follow-perturbation leader strategy aims to minimize cumulative loss. Its core principle is to explore the decision space by introducing random perturbations, and the update mechanism is based on... The implementation is achieved, where σ(π) is the perturbation added to the cumulative loss, and t is the time step.

Images