A self-supervised multi-scale cloud workload prediction method and system

Abstract

The application provides a kind of self-supervision multi-scale cloud workload prediction method and system in the technical field of cloud computing resource management, method includes: step S1, the time series data of each physical machine workload in cloud environment is collected;Step S2, each time series data is subjected to data enhancement operation to obtain enhanced sequence data;Step S3, based on multi-scale patch embedding layer and scale perception layer, multi-scale encoder is constructed;Step S4, based on enhanced sequence data, the self-supervision training of multi-scale encoder is carried out;Step S5, in multi-scale encoder, prediction head is added, and the adapter of specific scale is introduced, obtain preset quantity of labeled data, based on each labeled data, the adapter parameters of prediction head and adapter are fine-tuned, and then cloud workload prediction model is constructed;Step S6, based on cloud workload prediction model, workload prediction is carried out.The application has the advantages that in the case of labeled data scarcity, the workload prediction accuracy is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of cloud computing resource management technology, and in particular to a self-supervised multi-scale cloud workload prediction method and system. Background Technology

[0002] With the widespread application and rapid development of cloud computing, accurate workload forecasting has become a key technical means to achieve efficient resource management, cost optimization, and service quality assurance. Through accurate workload forecasting, cloud systems can make proactive scaling decisions, avoid over-provisioning of resources, and effectively reduce the occurrence of Service Level Agreement (SLA) breaches.

[0003] In recent years, deep learning-based methods, especially the Transformer architecture, have achieved significant results in cloud workload prediction tasks. Multi-scale Transformer models, such as Informer, Autoformer, and FEDformer, have achieved state-of-the-art performance on multiple benchmark tasks due to their efficient ability to capture short-term fluctuations and long-term periodic patterns. However, such complex models typically rely on large amounts of high-quality labeled data for training, and obtaining such labeled data in real-world cloud environments still faces many challenges. The data labeling challenges in cloud workload prediction mainly manifest in the following aspects:

[0004] 1. High Labeling Costs: Unlike traditional supervised learning tasks, cloud workload data requires continuous monitoring and precise measurement of resource utilization metrics. This process demands significant infrastructure investment and ongoing operational costs. Furthermore, obtaining accurate labels for future workloads requires waiting for actual consumption, resulting in significant time delays and further increasing the complexity of model training.

[0005] 2. Highly heterogeneous cloud environments: Different cloud service providers, data centers, and even within a single cluster exhibit drastically different workload characteristics due to differences in hardware configuration, application type, and user behavior. Predictive models trained in a specific environment often struggle to generalize directly to other scenarios; therefore, data collection and labeling must be performed separately for each deployment environment.

[0006] 3. Dynamic Evolution of Workload: Cloud applications and service models are rapidly iterating, with new services constantly being launched and existing services continuously updated. This dynamism makes historically labeled data easily outdated, requiring constant relabeling to maintain the predictive accuracy of the model.

[0007] 4. Privacy and Security Restrictions: Cloud workload data often contains sensitive information regarding user behavior and business operations. Due to privacy regulations and business competition considerations, organizations are typically unwilling to share detailed workload traces, which severely limits the public availability of high-quality labeled datasets.

[0008] 5. Class Imbalance Issue: Cloud workload data generally exhibits a highly skewed distribution, with normal operating conditions making up the vast majority of data, while critical events (such as load peaks or failures) occur very infrequently. This imbalance makes it extremely difficult to obtain effective labeled samples for rare patterns.

[0009] To address these challenges, the machine learning community is increasingly focusing on self-supervised learning (SSL), viewing it as an effective solution to reduce or even replace manual annotation. SSL, by designing pre-training tasks, learns transferable feature representations from unlabeled data and has demonstrated significant potential in a range of time series analysis tasks.

[0010] However, applying SSL to cloud workload forecasting still presents unique challenges: cloud workload time series typically exhibit complex multi-scale characteristics, encompassing fluctuations ranging from minute-level to day-level periodic changes. Existing general-purpose time series SSL methods often struggle to adequately capture the patterns specific to this domain. Furthermore, the high noise and irregular sampling issues commonly found in cloud monitoring data further degrade the quality of feature representations obtained through self-supervised learning.

[0011] Therefore, how to provide a self-supervised multi-scale cloud workload prediction method and system to improve workload prediction accuracy under the condition of scarce labeled data has become an urgent technical problem to be solved. Summary of the Invention

[0012] The technical problem to be solved by this invention is to provide a self-supervised multi-scale cloud workload prediction method and system, which improves the accuracy of workload prediction when labeled data is scarce.

[0013] In a first aspect, the present invention provides a self-supervised multi-scale cloud workload prediction method, comprising the following steps:

[0014] Step S1: Collect time-series data of the workload of each physical machine in the cloud environment. The time-series data includes at least CPU utilization, memory usage, network bandwidth usage, and disk I / O speed.

[0015] Step S2: Perform data augmentation operations on each of the time series data, including at least multi-scale time warping, frequency domain mixing, and periodic pattern injection, to obtain augmented sequence data;

[0016] Step S3: Based on the multi-scale patch embedding layer and the scale-aware layer, construct a multi-scale encoder for mapping the enhanced sequence data into a multi-scale temporal representation;

[0017] Step S4: Using a hierarchical contrastive learning framework, a multi-scale encoder is trained under self-supervised conditions based on the enhanced sequence data. During the training process, multi-scale contrastive loss and cross-scale consistency loss are combined.

[0018] Step S5: Add a prediction head to the multi-scale encoder after self-supervised training, introduce an adapter of a specific scale, obtain a preset number of labeled data, fine-tune the adapter parameters of the prediction head and the adapter based on the labeled data, and then construct a cloud workload prediction model.

[0019] Step S6: Perform workload prediction based on the cloud workload prediction model.

[0020] Furthermore, in step S2, the multi-scale time warp specifically refers to:

[0021] The time series data is decomposed using discrete wavelet transform. K Each scale component: ;

[0022] Where x represents time series data; Indicates the first K The scale component, i.e., the first scale component K Detail coefficients at each scale; Represents the approximation coefficient at the coarsest scale; k Indicates scale level;

[0023] Distort each of the aforementioned scale components: ;

[0024] in, Indicates the distorted ; This represents the torsion intensity control parameter, used to ensure that finer scales accept less torsion in order to maintain short-term patterns; , Indicates the basic torsional strength; Represents the twist function;

[0025] Based on the above as well as Construct time series data with multi-scale time warping: ;

[0026] in, Represents time-series data after multi-scale time warping;

[0027] The frequency domain mixing specifically refers to:

[0028] Acquire first time series data and second time series data, and perform Fourier transform on the first time series data and second time series data to obtain a first frequency domain representation and a second frequency domain representation:

[0029] ;

[0030] ;

[0031] in, This represents the first time series data; This represents the second time series data; This represents the first frequency domain representation; This represents the second frequency domain representation; Represents the Fourier transform operator;

[0032] Identify the set of periodic frequencies of the workloads in the first and second time series data: ;

[0033] in, Represents a set of periodic frequencies; Indicates the daily cycle frequency; Indicates the periodic frequency; Indicates the monthly cycle frequency;

[0034] Frequency mixing based on a mixing mask function: ;

[0035] in, Represents a mixing mask function; f Indicates frequency; This indicates Bernoulli random sampling; p Represents the probability of a random mixture;

[0036] Performing a spectrum enhancement operation on the first frequency domain representation and the second frequency domain representation yields a mixed frequency domain signal: ;

[0037] in, Represents a mixed frequency domain signal; This represents element-wise multiplication;

[0038] Perform an inverse Fourier transform on the mixed frequency domain signal to obtain the time series data after frequency domain mixing;

[0039] ;

[0040] in, This represents time series data after frequency domain mixing; This represents the inverse Fourier transform operator;

[0041] The periodic pattern injection specifically refers to:

[0042] Based on domain knowledge of workloads in the cloud environment, inject synthesized lifecycle patterns: ;

[0043] in, express t The periodic pattern of time; t Indicates the position of a time point in time series data; Indicates the number of injected periodic patterns; This represents the amplitude of the i-th periodic pattern; This represents the frequency of the i-th periodic pattern; This represents the phase of the i-th periodic pattern; Indicates the temporal localization window function;

[0044] Perform data augmentation operations on the time series data based on the aforementioned periodic pattern: ;

[0045] in, This represents time series data after periodic pattern injection; This represents the injection intensity coefficient for the periodic pattern.

[0046] Furthermore, in step S3, the multi-scale patch embedding layer is used to divide the augmented sequence data into patches of different sizes and embed them: ;

[0047] in, This represents a multi-scale patch embedding function; This represents augmented sequence data; This indicates a multi-scale patch splicing operation; Indicates the first S Scale-based embedding layer; Indicates the first S The size of the patch according to the scale;

[0048] The scale-aware layer is used to perform cross-scale interactive learning on each patch through several layers of Transformer blocks to obtain a multi-scale time representation.

[0049] Furthermore, in step S4, the total loss function of the multi-scale encoder is: ;

[0050] in, This represents the total loss value of the multi-scale encoder; Indicates multi-scale contrast loss; This represents the cross-scale consistency loss; Indicates the scale number; as well as Both represent loss balance coefficients.

[0051] Furthermore, in step S5, the formula for the prediction head is: ;

[0052] in, Indicates the prediction head; Indicates the prediction head parameters; Represent real numbers; S Indicates the scale number; D Indicates dimension; Indicates the predicted time span;

[0053] The formula for the aggregated multi-scale time representation of the prediction head is: ;

[0054] in, This indicates the output of the prediction head; Representing scale S Attention weights; Representing scale S Dedicated multilayer sensor; Indicates the first S Scale ;

[0055] The formula for the adapter is: ;

[0056] in, Indicates the first S Scale adapter function; Indicates the hidden state of the input adapter; Indicates the activation function; Represents a dimension-reduced matrix; This represents an upgraded matrix.

[0057] Secondly, this invention provides a self-supervised multi-scale cloud workload prediction system, comprising the following modules:

[0058] The time series data acquisition module is used to collect time series data of the workload of each physical machine in the cloud environment. The time series data includes at least CPU utilization, memory usage, network bandwidth usage, and disk I / O speed.

[0059] The data augmentation module is used to perform data augmentation operations on each of the time series data, including at least multi-scale time warping, frequency domain mixing, and periodic pattern injection, to obtain augmented sequence data;

[0060] A multi-scale encoder construction module is used to construct a multi-scale encoder that maps the enhanced sequence data into a multi-scale temporal representation based on a multi-scale patch embedding layer and a scale-aware layer.

[0061] The self-supervised training module is used to perform self-supervised training of the multi-scale encoder based on the augmented sequence data using a hierarchical contrastive learning framework, combining multi-scale contrastive loss and cross-scale consistency loss during the training process.

[0062] The fine-tuning module is used to add a prediction head to the multi-scale encoder after self-supervised training, introduce an adapter of a specific scale, obtain a preset number of labeled data, and fine-tune the adapter parameters of the prediction head and the adapter based on the labeled data, thereby constructing a cloud workload prediction model.

[0063] The workload prediction module is used to predict workloads based on the cloud workload prediction model.

[0064] Furthermore, in the data augmentation module, the multi-scale time warp specifically refers to:

[0065] The time series data is decomposed using discrete wavelet transform. K Each scale component: ;

[0066] Where x represents time series data; Indicates the first K The scale component, i.e., the first scale component K Detail coefficients at each scale; Represents the approximation coefficient at the coarsest scale; k Indicates scale level;

[0067] Distort each of the aforementioned scale components: ;

[0068] in, Indicates the distorted ; This represents the torsion intensity control parameter, used to ensure that finer scales accept less torsion in order to maintain short-term patterns; , Indicates the basic torsional strength; Represents the twist function;

[0069] Based on the above as well as Construct time series data with multi-scale time warping: ;

[0070] in, Represents time-series data after multi-scale time warping;

[0071] The frequency domain mixing specifically refers to:

[0072] Acquire first time series data and second time series data, and perform Fourier transform on the first time series data and second time series data to obtain a first frequency domain representation and a second frequency domain representation: ;

[0073] ;

[0074] in, This represents the first time series data; This represents the second time series data; This represents the first frequency domain representation; This represents the second frequency domain representation; Represents the Fourier transform operator;

[0075] Identify the set of periodic frequencies of the workloads in the first and second time series data: ;

[0076] in, Represents a set of periodic frequencies; Indicates the daily cycle frequency; Indicates the periodic frequency; Indicates the monthly cycle frequency;

[0077] Frequency mixing based on a mixing mask function: ;

[0078] in, Represents a mixing mask function; f Indicates frequency; This indicates Bernoulli random sampling; p Represents the probability of a random mixture;

[0079] Performing a spectrum enhancement operation on the first frequency domain representation and the second frequency domain representation yields a mixed frequency domain signal: ;

[0080] in, Represents a mixed frequency domain signal; This represents element-wise multiplication;

[0081] Perform an inverse Fourier transform on the mixed frequency domain signal to obtain the time series data after frequency domain mixing; ;

[0082] in, This represents time series data after frequency domain mixing; This represents the inverse Fourier transform operator;

[0083] The periodic pattern injection specifically refers to:

[0084] Based on domain knowledge of workloads in the cloud environment, inject synthesized lifecycle patterns: ;

[0085] in, express t The periodic pattern of time; t Indicates the position of a time point in time series data; Indicates the number of injected periodic patterns; This represents the amplitude of the i-th periodic pattern; This represents the frequency of the i-th periodic pattern; This represents the phase of the i-th periodic pattern; Indicates the temporal localization window function;

[0086] Perform data augmentation operations on the time series data based on the aforementioned periodic pattern: ;

[0087] in, This represents time series data after periodic pattern injection; This represents the injection intensity coefficient for the periodic pattern.

[0088] Furthermore, in the multi-scale encoder construction module, the multi-scale patch embedding layer is used to divide the augmented sequence data into patches of different sizes and embed them: ;

[0089] in, This represents a multi-scale patch embedding function; This represents augmented sequence data; This indicates a multi-scale patch splicing operation; Indicates the first S Scale-based embedding layer; Indicates the first S The size of the patch according to the scale;

[0090] The scale-aware layer is used to perform cross-scale interactive learning on each patch through several layers of Transformer blocks to obtain a multi-scale time representation.

[0091] Furthermore, in the self-supervised training module, the total loss function of the multi-scale encoder is: ;

[0092] in, This represents the total loss value of the multi-scale encoder; Indicates multi-scale contrast loss; This represents the cross-scale consistency loss; Indicates the scale number; as well as Both represent loss balance coefficients.

[0093] Furthermore, in the fine-tuning module, the formula for the prediction head is: ;

[0094] in, Indicates the prediction head; Indicates the prediction head parameters; Represent real numbers; Indicates the scale number; D Indicates dimension; Indicates the predicted time span;

[0095] The formula for the aggregated multi-scale time representation of the prediction head is: ;

[0096] in, This indicates the output of the prediction head; Representing scale S Attention weights; Representing scale S Dedicated multilayer sensor; Indicates the first S Scale ;

[0097] The formula for the adapter is: ;

[0098] in, Indicates the first S Scale adapter function; Indicates the hidden state of the input adapter; Indicates the activation function; Represents a dimension-reduced matrix; This represents an upgraded matrix.

[0099] The advantages of this invention are:

[0100] 1. By collecting time-series data of the workload of each physical machine in the cloud environment, including at least CPU utilization, memory usage, network bandwidth usage, and disk I / O speed, data augmentation operations, including at least multi-scale time warping, frequency domain mixing, and periodic pattern injection, are performed on each time-series data to obtain augmented sequence data. Then, a multi-scale encoder is constructed based on a multi-scale patch embedding layer and a scale-aware layer to map the augmented sequence data to a multi-scale time representation. A hierarchical contrastive learning framework is used to perform self-supervised training of the multi-scale encoder based on the augmented sequence data, incorporating multi-scale contrastive loss and cross-scale consistency loss during training. A prediction head is added to the self-supervised multi-scale encoder, and an adapter of a specific scale is introduced to obtain a preset number of labeled data. Based on each labeled data... By fine-tuning the parameters of the prediction head and adapter, a cloud workload prediction model is constructed, and finally, workload prediction is performed based on the cloud workload prediction model. Specifically, a multi-scale encoder is pre-trained on unlabeled data (augmented sequence data) using a hierarchical contrastive learning framework, combined with domain knowledge-driven data augmentation (such as multi-scale time warping to protect short-term fluctuations, frequency domain hybridization to preserve periodic patterns, and synthetic periodic injection to enhance domain features). This enables the multi-scale encoder to automatically learn the multi-scale representation (multi-scale time representation) of cloud workloads from massive amounts of unlabeled monitoring data. Subsequently, only a lightweight prediction head and adapter need to be introduced, and the prediction head and adapter parameters can be fine-tuned using a very small amount of labeled data to quickly adapt to specific environments. Ultimately, this greatly improves the accuracy of workload prediction even when labeled data is scarce.

[0101] 2. This invention achieves near-fully supervised learning prediction performance using only 20% of the labeled data, significantly reducing reliance on large-scale data labeling. It also possesses strong cross-cloud environment transfer learning capabilities, effectively improving the accuracy and timeliness of cloud computing resource scheduling, and drastically reducing data labeling costs and time.

[0102] 3. Data augmentation strategies and multi-scale architectures designed for cloud workload characteristics can effectively capture multi-scale time patterns from minute to week, thereby improving workload prediction accuracy.

[0103] 4. The pre-trained representations have good versatility and can be effectively transferred between different cloud environments, improving prediction performance by 2.4%-5.2%; it supports efficient fine-tuning (12 minutes) and low-memory deployment (2.1GB), making it suitable for real-time cloud environment deployment; the multi-scale attention mechanism provides an interpretable decision-making process, enhancing model transparency and credibility.

[0104] 5. By collecting time-series data of cloud workloads, applying cloud-specific data augmentation strategies to generate diverse training samples, constructing a multi-scale encoder to extract multi-scale time representations, employing hierarchical contrastive learning for self-supervised pre-training, and finally achieving accurate predictions with limited labeled data through efficient fine-tuning; that is, the augmentation strategy designed based on cloud workload characteristics can maintain important time patterns while increasing data diversity; the multi-scale architecture can simultaneously capture hierarchical time dependencies from hourly to weekly levels; the contrastive learning framework learns robust representations in an unsupervised environment; the efficient fine-tuning strategy prevents catastrophic forgetting and quickly adapts to the target task; combined with cross-cloud environment transfer learning capabilities, the model can be effectively reused across different cloud platforms, ultimately significantly reducing label dependencies, improving prediction accuracy, and lowering deployment costs, providing a practical prediction solution for cloud computing resource management. Attached Figure Description

[0105] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0106] Figure 1 This is a flowchart of a self-supervised multi-scale cloud workload prediction method according to the present invention.

[0107] Figure 2 This is a schematic diagram of the structure of a self-supervised multi-scale cloud workload prediction system according to the present invention.

[0108] Figure 3 This is a flowchart illustrating the present invention.

[0109] Figure 4 This is a schematic diagram of the multi-scale time warp of the present invention.

[0110] Figure 5 This is a schematic diagram of the multi-scale encoder of the present invention.

[0111] Figure 6 This is a schematic diagram of the fine-tuning comparison of the present invention.

[0112] Figure 7-9 This is a performance comparison chart of the present invention under different proportions of labeled data.

[0113] Figure 10-12 This is a graph showing the effect analysis of cross-dataset transfer learning in this invention. Detailed Implementation

[0114] The overall idea of ​​the technical solution in this application is as follows: By pre-training a multi-scale encoder on unlabeled data using a hierarchical contrastive learning framework, and combining it with domain knowledge-driven data augmentation, the multi-scale encoder can automatically learn the multi-scale representation of cloud workloads from massive amounts of unlabeled monitoring data. Subsequently, only a lightweight prediction head and adapter need to be introduced, and the prediction head and adapter parameters can be fine-tuned using a very small amount of labeled data to quickly adapt to the specific environment, thereby improving the workload prediction accuracy when labeled data is scarce.

[0115] Please refer to Figures 1 to 12 As shown, Figure 4 The multi-scale time warp (MSTW) process is described in detail: (a) the original cloud workload time series; (b) multiple scale components (d1, d2, d3, a3) obtained through wavelet decomposition; (c) applying warps of different intensities to each scale component. (d) Comparison of the reconstructed enhanced sequence with the original sequence; Figure 5 The hierarchical architecture of the multi-scale encoder is shown: the input time series is divided into multi-scale patches (p1=8, p2=16, p3=32, p4=64), each scale patch is processed by the embedding layer (E1-E4), and then passed through L=6 scale-aware Transformer blocks. Finally, the multi-scale representation is generated by scale attention aggregation. Figure 6 Two fine-tuning strategies were compared: (a) a full fine-tuning strategy updates all pre-trained encoder parameters and the prediction head, with a training parameter count of O(LD2); (b) an efficient fine-tuning strategy freezes the pre-trained encoder and updates only the lightweight adapter and the prediction head, with a training parameter count of O(Sd·aD), where... ; Figure 7-9 The performance of this invention under different proportions of labeled data is shown. The performance of this invention and other methods under different proportions of labeled data (5%-100%) is compared on three datasets: Alibaba, Google and Azure. The results show that this invention can achieve performance close to the Pathformer fully supervised method (100% labeled data) using only 20% labeled data. Figure 10-12 The strong transfer learning capability of the present invention is demonstrated: (a) the similarity matrix between datasets shows high similarity between Alibaba-Google (0.82), Alibaba-Azure (0.75), and Google-Azure (0.78); (b) the relationship between performance gain and dataset similarity; (c) the evolution of feature alignment at each layer during fine-tuning, showing a pattern of rapid adaptation at the bottom layer and gradual optimization at the top layer.

[0116] A preferred embodiment of the self-supervised multi-scale cloud workload prediction method of the present invention includes the following steps:

[0117] Step S1: Collect time-series data of the workload of each physical machine in the cloud environment. The time-series data includes at least CPU utilization, memory usage, network bandwidth usage, and disk I / O speed.

[0118] In practice, a sliding window strategy is used to collect historical data (time series data). The window size is set to 336 time steps, corresponding to 7 days of historical data (5-minute sampling interval) or 14 days of historical data (1-hour sampling interval). Robust scaling is used for data preprocessing. ;

[0119] Wherein, IQR stands for interquartile range; robust scaling is robust to outliers commonly found in cloud workloads;

[0120] Step S2: Perform data augmentation operations on each of the time series data, including at least multi-scale time warp (MSTW), frequency domain mixing (FDM), and periodic pattern injection (PPI), to obtain augmented sequence data;

[0121] Multi-scale time warping is used to decompose time series data through discrete wavelet transform and perform scale-specific warping on each scale component; frequency domain mixing is used to selectively mix different time series data in the frequency domain to maintain periodic characteristics; periodic pattern injection is used to synthesize periodic patterns based on domain knowledge injection from cloud workloads.

[0122] Step S3: Based on the multi-scale patch embedding layer and the scale-aware layer, construct a multi-scale encoder for mapping the enhanced sequence data to a multi-scale temporal representation; the formula for the multi-scale encoder is expressed as: ;

[0123] in, This indicates a multi-scale encoder; Represent real numbers; Indicates the total length of the augmented sequence data; S Indicates the scale number; D Indicates dimension;

[0124] The multi-scale encoder adopts a hierarchical architecture, which divides the input into patches of different sizes through multi-scale patch embedding and uses a scale-aware attention mechanism to achieve cross-scale information exchange, enabling it to capture both fine-grained changes and long-term trends simultaneously.

[0125] In practice, the output of the multi-scale encoder is mapped to a low-dimensional space suitable for contrastive learning through a projection network: ;

[0126] in, Represents a projection network; Indicates multi-scale time representation; as well as Represents the weight matrix; as well as Both represent bias vectors;

[0127] Step S4: Using a hierarchical contrastive learning framework, a multi-scale encoder is trained under self-supervised conditions based on the enhanced sequence data. During the training process, multi-scale contrastive loss and cross-scale consistency loss are combined.

[0128] The hierarchical contrastive learning framework performs contrastive learning simultaneously at multiple time scales. It optimizes multi-scale representations through multi-scale contrastive loss and cross-scale consistency loss, enabling the model to learn hierarchical time patterns from hourly to weekly levels.

[0129] Step S5: Add a prediction head to the multi-scale encoder after self-supervised training, introduce an adapter of a specific scale, obtain a preset number of labeled data, and fine-tune the adapter parameters of the prediction head and the adapter based on the labeled data to construct a cloud workload prediction model; that is, use a small amount of labeled data for efficient fine-tuning.

[0130] The efficient fine-tuning strategy freezes most of the pre-trained parameters and only updates the lightweight adapter and prediction head. It prevents catastrophic forgetting through a progressive fine-tuning strategy and achieves rapid adaptation with a small amount of labeled data.

[0131] Step S6: Perform workload prediction based on the cloud workload prediction model, that is, output cloud workload prediction results for multiple time spans through the cloud workload prediction model, supporting short-term and medium-to-long-term predictions.

[0132] The cloud workload prediction model supports multiple prediction time spans H∈{24,48,96,192}, corresponding to short-term (1-2 days) to medium-term (4-8 days) prediction scenarios. It supports long-sequence processing through sliding window inference without linearly increasing memory usage.

[0133] In step S2, the multi-scale time warp specifically refers to:

[0134] The time series data is decomposed using discrete wavelet transform. K Each scale component: ;

[0135] Where x represents time series data; Indicates the first K The scale component, i.e., the first scale component K Detail coefficients at each scale; Represents the approximation coefficient at the coarsest scale; k Indicates scale level;

[0136] Scale-specific distortion is applied to each of the aforementioned scale components: ;

[0137] in, Indicates the distorted ; This represents the torsion intensity control parameter, used to ensure that finer scales accept less torsion in order to maintain short-term patterns; , Indicates the basic torsional strength; Represents the twist function;

[0138] Based on the above as well as Construct time series data with multi-scale time warping: ;

[0139] in, Represents time-series data after multi-scale time warping;

[0140] The formula for the twist function is: ;

[0141] in, t Indicates the position of a time point in time series data; M The number of terms in the Fourier series determines the complexity of the distortion function; the larger the value, the more high-frequency disturbances there are, and the smaller the value, the smoother the distortion. Represents the random amplitude coefficient, ensuring smoothness. ,variance Ensure that the amplitude of high-frequency terms (large i) decays to avoid abrupt disturbances; i represents the order index of the Fourier series; Indicates the total length of the time series data;

[0142] Traditional time warping can disrupt important periodic patterns in cloud workloads. Therefore, this invention proposes a multi-scale time warping strategy that preserves periodic patterns at different time granularities.

[0143] The frequency domain mixing specifically refers to:

[0144] Acquire first time series data and second time series data, and perform Fourier transform on the first time series data and second time series data to obtain a first frequency domain representation and a second frequency domain representation:

[0145] ;

[0146] ;

[0147] in, This represents the first time series data; This represents the second time series data; This represents the first frequency domain representation; This represents the second frequency domain representation; Represents the Fourier transform operator;

[0148] Identify the set of periodic frequencies of the workloads in the first and second time series data: ;

[0149] in, Represents a set of periodic frequencies; Indicates the daily cycle frequency; Indicates the periodic frequency; Indicates the monthly cycle frequency;

[0150] Frequency mixing based on a mixing mask function: ;

[0151] in, Represents a mixing mask function; f Indicates frequency; This indicates Bernoulli random sampling; p Represents the probability of a random mixture;

[0152] Performing a spectrum enhancement operation on the first frequency domain representation and the second frequency domain representation yields a mixed frequency domain signal: ;

[0153] in, Represents a mixed frequency domain signal; This represents element-wise multiplication;

[0154] Perform an inverse Fourier transform on the mixed frequency domain signal to obtain the time series data after frequency domain mixing; ;

[0155] in, This represents time series data after frequency domain mixing; This represents the inverse Fourier transform operator;

[0156] Cloud workloads typically exhibit strong cyclical patterns (daily and weekly cycles), and frequency domain hybrid strategies are designed to introduce controlled variations while maintaining these cyclicalities.

[0157] The periodic pattern injection specifically refers to:

[0158] Based on domain knowledge of workloads in the cloud environment, inject synthesized lifecycle patterns: ;

[0159] in, express t The periodic pattern of time; tIndicates the position of a time point in time series data; Indicates the number of injected periodic patterns; This represents the amplitude of the i-th periodic pattern; This represents the frequency of the i-th periodic pattern; This represents the phase of the i-th periodic pattern; Indicates the temporal localization window function;

[0160] Perform data augmentation operations on the time series data based on the aforementioned periodic pattern: ;

[0161] in, This represents time series data after periodic pattern injection; This represents the injection intensity coefficient for the periodic pattern, typically ranging from 0.1 to 0.3.

[0162] In step S3, the multi-scale patch embedding layer is used to divide the augmented sequence data into patches of different sizes and embed them: ;

[0163] in, This represents a multi-scale patch embedding function; This represents augmented sequence data; This indicates a multi-scale patch splicing operation; Indicates the first S Scale-based embedding layer; Indicates the first S The size of the patch according to the scale; , Indicates the base patch size. s Indicates scale index;

[0164] The scale-aware layer is used to perform cross-scale interactive learning on each patch through several layers of Transformer blocks to obtain a multi-scale time representation.

[0165] The formula for the scale-aware attention mechanism of the Transformer block is: ;

[0166] in, express Multiscale temporal representation, i.e., scale-aware attentional output; This represents the output of the multi-scale patch embedding layer; This represents the normalized exponential function; Represents the query matrix; Represents the key matrix; Represents a value matrix; Indicates transpose; Represents the scale-aware bias matrix; Indicates the dimension of the key vector or query vector.

[0167] No. The output of the nth Transformer block is the nth Inputs to a Transformer block: .

[0168] In step S4, the total loss function of the multi-scale encoder is: ;

[0169] in, This represents the total loss value of the multi-scale encoder; Indicates multi-scale contrast loss; This represents the cross-scale consistency loss; Indicates the scale number; as well as Both represent loss balance coefficients. ;

[0170] ;

[0171] in, N Indicates the batch size of the augmented sequence data; exp() represents the natural exponential function; sim(,) represents the cosine similarity function; This represents the first augmented view of the i-th augmented sequence data at scale. S The representation vector; This represents the second augmented view of the i-th augmented sequence data at scale. S The representation vector; This represents the second augmented view of the j-th augmented sequence data at scale. S The representation vector; Represents the representation vector of the first augmented view of the i-th augmented sequence data at scale S+1; Indicates temperature parameter; This indicates an indicator function that takes the value 1 when j ≠ i, and 0 otherwise. This represents the alignment operation function; This represents the square of the L2 norm.

[0172] The learning mechanism of the hierarchical contrastive learning framework is as follows:

[0173] For a given batch N Two augmented views are generated from the augmented sequence data: .

[0174] In step S5, the formula for the prediction head is: ;

[0175] in, Indicates the prediction head; Indicates the prediction head parameters; Represent real numbers; S Indicates the scale number; D Indicates dimension; Indicates the predicted time span;

[0176] The formula for the aggregated multi-scale time representation of the prediction head is: ;

[0177] in, This indicates the output of the prediction head; Representing scale S Attention weights; Representing scale S Dedicated multilayer sensor; Indicates the first S The time span for scale prediction;

[0178] ;

[0179] Where w represents the learnable scale importance vector; T Indicates transpose; Show the j-th scale ;

[0180] The formula for the adapter is: ;

[0181] in, Indicates the first S Scale adapter function; Indicates the hidden state of the input adapter; Indicates the activation function; Represents a dimension-reduced matrix; This represents an upgraded matrix.

[0182] The fine-tuning loss during the fine-tuning process combines the prediction error loss with regularization: ;

[0183] in, Indicates fine-tuning loss; This represents the labeled data, i.e., the actual labels; Indicates the fine-tuning coefficient; This indicates the parameters after fine-tuning; This represents the pre-trained parameters.

[0184] For better stability, a progressive fine-tuning strategy is adopted: Stage 1: Train only the prediction head, train for E1 rounds; Stage 2: Unfreeze the top L / 3 encoder layer, train for E2 rounds; Stage 3: Unfreeze all layers, train for E3 rounds with a smaller learning rate.

[0185] A preferred embodiment of the self-supervised multi-scale cloud workload prediction system of the present invention includes the following modules:

[0186] The time series data acquisition module is used to collect time series data of the workload of each physical machine in the cloud environment. The time series data includes at least CPU utilization, memory usage, network bandwidth usage, and disk I / O speed.

[0187] In practice, a sliding window strategy is used to collect historical data (time series data). The window size is set to 336 time steps, corresponding to 7 days of historical data (5-minute sampling interval) or 14 days of historical data (1-hour sampling interval). Robust scaling is used for data preprocessing. ;

[0188] Wherein, IQR stands for interquartile range; robust scaling is robust to outliers commonly found in cloud workloads;

[0189] The data augmentation module is used to perform data augmentation operations on each of the time series data, including at least multi-scale time warp (MSTW), frequency domain mixing (FDM), and periodic pattern injection (PPI), to obtain augmented sequence data;

[0190] Multi-scale time warping is used to decompose time series data through discrete wavelet transform and perform scale-specific warping on each scale component; frequency domain mixing is used to selectively mix different time series data in the frequency domain to maintain periodic characteristics; periodic pattern injection is used to synthesize periodic patterns based on domain knowledge injection from cloud workloads.

[0191] A multi-scale encoder construction module is used to construct a multi-scale encoder that maps the enhanced sequence data to a multi-scale temporal representation based on a multi-scale patch embedding layer and a scale-aware layer; the formula for the multi-scale encoder is expressed as: ;

[0192] in, This indicates a multi-scale encoder; Represent real numbers; Indicates the total length of the augmented sequence data; S Indicates the scale number; D Indicates dimension;

[0193] The multi-scale encoder adopts a hierarchical architecture, which divides the input into patches of different sizes through multi-scale patch embedding and uses a scale-aware attention mechanism to achieve cross-scale information exchange, enabling it to capture both fine-grained changes and long-term trends simultaneously.

[0194] In practice, the output of the multi-scale encoder is mapped to a low-dimensional space suitable for contrastive learning through a projection network: ;

[0195] in, Represents a projection network; Indicates multi-scale time representation; as well as Both represent weight matrices; as well as Both represent bias vectors;

[0196] The self-supervised training module is used to perform self-supervised training of the multi-scale encoder based on the augmented sequence data using a hierarchical contrastive learning framework, combining multi-scale contrastive loss and cross-scale consistency loss during the training process.

[0197] The hierarchical contrastive learning framework performs contrastive learning simultaneously at multiple time scales. It optimizes multi-scale representations through multi-scale contrastive loss and cross-scale consistency loss, enabling the model to learn hierarchical time patterns from hourly to weekly levels.

[0198] The fine-tuning module is used to add a prediction head to the multi-scale encoder after self-supervised training, introduce an adapter of a specific scale, obtain a preset number of labeled data, and fine-tune the adapter parameters of the prediction head and the adapter based on the labeled data, thereby constructing a cloud workload prediction model; that is, to use a small amount of labeled data for efficient fine-tuning.

[0199] The efficient fine-tuning strategy freezes most of the pre-trained parameters and only updates the lightweight adapter and prediction head. It prevents catastrophic forgetting through a progressive fine-tuning strategy and achieves rapid adaptation with a small amount of labeled data.

[0200] The workload prediction module is used to predict workloads based on the cloud workload prediction model, that is, to output cloud workload prediction results for multiple time spans through the cloud workload prediction model, supporting short-term and medium-to-long-term predictions.

[0201] The cloud workload prediction model supports multiple prediction time spans H∈{24,48,96,192}, corresponding to short-term (1-2 days) to medium-term (4-8 days) prediction scenarios. It supports long-sequence processing through sliding window inference without linearly increasing memory usage.

[0202] In the data augmentation module, the multi-scale time warp specifically refers to:

[0203] The time series data is decomposed using discrete wavelet transform. K Each scale component: ;

[0204] Where x represents time series data; Indicates the first K The scale component, i.e., the first scale component K Detail coefficients at each scale; Represents the approximation coefficient at the coarsest scale; k Indicates scale level;

[0205] Scale-specific distortion is applied to each of the aforementioned scale components: ;

[0206] in, Indicates the distorted ; This represents the torsion intensity control parameter, used to ensure that finer scales accept less torsion in order to maintain short-term patterns; , Indicates the basic torsional strength; Represents the twist function;

[0207] Based on the above as well as Construct time series data with multi-scale time warping: ;

[0208] in, Represents time-series data after multi-scale time warping;

[0209] The formula for the twist function is: ;

[0210] in, t Indicates the position of a time point in time series data; M The number of terms in the Fourier series determines the complexity of the distortion function; the larger the value, the more high-frequency disturbances there are, and the smaller the value, the smoother the distortion. Represents the random amplitude coefficient, ensuring smoothness. ,variance Ensure that the amplitude of high-frequency terms (large i) decays to avoid abrupt disturbances; i represents the order index of the Fourier series; Indicates the total length of the time series data;

[0211] Traditional time warping can disrupt important periodic patterns in cloud workloads. Therefore, this invention proposes a multi-scale time warping strategy that preserves periodic patterns at different time granularities.

[0212] The frequency domain mixing specifically refers to:

[0213] Acquire first time series data and second time series data, and perform Fourier transform on the first time series data and second time series data to obtain a first frequency domain representation and a second frequency domain representation: ; ;

[0214] in, This represents the first time series data; This represents the second time series data; The first frequency domain representation is shown; This represents the second frequency domain representation; Represents the Fourier transform operator;

[0215] Identify the set of periodic frequencies of the workloads in the first and second time series data: ;

[0216] in, Represents a set of periodic frequencies; Indicates the daily cycle frequency; Indicates the periodic frequency; Indicates the monthly cycle frequency;

[0217] Frequency mixing based on a mixing mask function: ;

[0218] in, Represents a mixing mask function; f Indicates frequency; Bernoulli random sampling; p Represents the probability of a random mixture;

[0219] Performing a spectrum enhancement operation on the first frequency domain representation and the second frequency domain representation yields a mixed frequency domain signal: ;

[0220] in, Represents a mixed frequency domain signal; This represents element-wise multiplication;

[0221] Perform an inverse Fourier transform on the mixed frequency domain signal to obtain the time series data after frequency domain mixing; ;

[0222] in, This represents time series data after frequency domain mixing; This represents the inverse Fourier transform operator;

[0223] Cloud workloads typically exhibit strong cyclical patterns (daily and weekly cycles), and frequency domain hybrid strategies are designed to introduce controlled variations while maintaining these cyclicalities.

[0224] The periodic pattern injection specifically refers to:

[0225] Based on domain knowledge of workloads in the cloud environment, inject synthesized lifecycle patterns: ;

[0226] in, Show t The periodic pattern of time; t Indicates the position of a time point in time series data; Indicates the number of injected periodic patterns; This represents the amplitude of the i-th periodic pattern; This represents the frequency of the i-th periodic pattern; This represents the phase of the i-th periodic pattern; Indicates the temporal localization window function;

[0227] Perform data augmentation operations on the time series data based on the aforementioned periodic pattern: ;

[0228] in, This represents time series data after periodic pattern injection; This represents the injection intensity coefficient for the periodic pattern, typically ranging from 0.1 to 0.3.

[0229] In the multi-scale encoder construction module, the multi-scale patch embedding layer is used to divide the augmented sequence data into patches of different sizes and embed them: ;

[0230] in, This represents a multi-scale patch embedding function; This represents augmented sequence data; This indicates a multi-scale patch splicing operation; Indicates the first S Scale-based embedding layer; Indicates the first S The size of the patch according to the scale; , Indicates the base patch size. s Indicates scale index;

[0231] The scale-aware layer is used to perform cross-scale interactive learning on each patch through several layers of Transformer blocks to obtain a multi-scale time representation.

[0232] The formula for the scale-aware attention mechanism of the Transformer block is: ;

[0233] in, express Multiscale temporal representation, i.e., scale-aware attentional output; This represents the output of the multi-scale patch embedding layer; This represents the normalized exponential function; Represents the query matrix; Represents the key matrix; Represents a value matrix; Indicates transpose; Represents the scale-aware bias matrix; Indicates the dimension of the key vector or query vector.

[0234] No. The output of the nth Transformer block is the nth Inputs to a Transformer block: .

[0235] In the self-supervised training module, the total loss function of the multi-scale encoder is: ;

[0236] in, This represents the total loss value of the multi-scale encoder; Indicates multi-scale contrast loss; This represents the cross-scale consistency loss; Indicates the scale number; as well as Both represent loss balance coefficients. ;

[0237] ;

[0238] in, N Indicates the batch size of the augmented sequence data; exp() represents the natural exponential function; sim(,) represents the cosine similarity function; This represents the first augmented view of the i-th augmented sequence data at scale. S The representation vector; This represents the second augmented view of the i-th augmented sequence data at scale. S The representation vector; This represents the second augmented view of the j-th augmented sequence data at scale. S The representation vector; Represents the representation vector of the first augmented view of the i-th augmented sequence data at scale S+1; Indicates temperature parameter; This indicates an indicator function that takes the value 1 when j ≠ i, and 0 otherwise. This represents the alignment operation function; This represents the square of the L2 norm.

[0239] The learning mechanism of the hierarchical contrastive learning framework is as follows:

[0240] For a given batch N Two augmented views are generated from the augmented sequence data: .

[0241] In the fine-tuning module, the formula for the prediction head is: ;

[0242] in, Indicates the prediction head; Indicates the prediction head parameters; Represent real numbers; S Indicates the scale number; D Indicates dimension; Indicates the predicted time span;

[0243] The formula for the aggregated multi-scale time representation of the prediction head is: ;

[0244] in, This indicates the output of the prediction head; Representing scale S Attention weights; Representing scale S Dedicated multilayer sensor; Indicates the first S The time span for scale prediction;

[0245] ;

[0246] Where w represents the learnable scale importance vector; T Indicates transpose; Represents the j-th scale ;

[0247] The formula for the adapter is: ;

[0248] in, Indicates the first S Scale adapter function; Indicates the hidden state of the input adapter; Indicates the activation function; Represents a dimension-reduced matrix; This represents an upgraded matrix.

[0249] The fine-tuning loss during the fine-tuning process combines the prediction error loss with regularization: ;

[0250] in, Indicates fine-tuning loss; This represents the labeled data, i.e., the actual labels; Indicates the fine-tuning coefficient; This indicates the parameters after fine-tuning; This represents the pre-trained parameters.

[0251] For better stability, a progressive fine-tuning strategy is adopted: Stage 1: Train only the prediction head, train for E1 rounds; Stage 2: Unfreeze the top L / 3 encoder layer, train for E2 rounds; Stage 3: Unfreeze all layers, train for E3 rounds with a smaller learning rate.

[0252] Experimental results demonstrate that this invention achieves significant performance on Alibaba cluster trajectories, Google cluster trajectories, and Azure public datasets: 1. It achieves near-fully supervised baseline performance using only 20% labeled data, reducing RMSE from 0.381 to 0.335 on the Alibaba dataset, a 12.1% improvement; 2. It exhibits strong transfer learning capabilities, achieving a 2.4%-5.2% performance improvement across datasets; 3. Pre-training time is 24.1 hours, fine-tuning time is only 12 minutes (high-efficiency version), inference latency is 44ms / batch, and memory consumption is 2.1GB (high-efficiency version); 4. It maintains good performance even in scenarios with extremely small labeled data (5% labeled data), achieving a 23-26% improvement over supervised learning methods.

[0253] In summary, the advantages of this invention are as follows:

[0254] 1. By collecting time-series data of the workload of each physical machine in the cloud environment, including at least CPU utilization, memory usage, network bandwidth usage, and disk I / O speed, data augmentation operations, including at least multi-scale time warping, frequency domain mixing, and periodic pattern injection, are performed on each time-series data to obtain augmented sequence data. Then, a multi-scale encoder is constructed based on a multi-scale patch embedding layer and a scale-aware layer to map the augmented sequence data to a multi-scale time representation. A hierarchical contrastive learning framework is used to perform self-supervised training of the multi-scale encoder based on the augmented sequence data, incorporating multi-scale contrastive loss and cross-scale consistency loss during training. A prediction head is added to the self-supervised multi-scale encoder, and an adapter of a specific scale is introduced to obtain a preset number of labeled data. Based on each labeled data... By fine-tuning the parameters of the prediction head and adapter, a cloud workload prediction model is constructed, and finally, workload prediction is performed based on the cloud workload prediction model. Specifically, a multi-scale encoder is pre-trained on unlabeled data (augmented sequence data) using a hierarchical contrastive learning framework, combined with domain knowledge-driven data augmentation (such as multi-scale time warping to protect short-term fluctuations, frequency domain hybridization to preserve periodic patterns, and synthetic periodic injection to enhance domain features). This enables the multi-scale encoder to automatically learn the multi-scale representation (multi-scale time representation) of cloud workloads from massive amounts of unlabeled monitoring data. Subsequently, only a lightweight prediction head and adapter need to be introduced, and the prediction head and adapter parameters can be fine-tuned using a very small amount of labeled data to quickly adapt to specific environments. Ultimately, this greatly improves the accuracy of workload prediction even when labeled data is scarce.

[0255] 2. This invention achieves near-fully supervised learning prediction performance using only 20% of the labeled data, significantly reducing reliance on large-scale data labeling. It also possesses strong cross-cloud environment transfer learning capabilities, effectively improving the accuracy and timeliness of cloud computing resource scheduling, and drastically reducing data labeling costs and time.

[0256] 3. Data augmentation strategies and multi-scale architectures designed for cloud workload characteristics can effectively capture multi-scale time patterns from minute to week, thereby improving workload prediction accuracy.

[0257] 4. The pre-trained representations have good versatility and can be effectively transferred between different cloud environments, improving prediction performance by 2.4%-5.2%; it supports efficient fine-tuning (12 minutes) and low-memory deployment (2.1GB), making it suitable for real-time cloud environment deployment; the multi-scale attention mechanism provides an interpretable decision-making process, enhancing model transparency and credibility.

[0258] 5. By collecting time-series data of cloud workloads, applying cloud-specific data augmentation strategies to generate diverse training samples, constructing a multi-scale encoder to extract multi-scale time representations, employing hierarchical contrastive learning for self-supervised pre-training, and finally achieving accurate predictions with limited labeled data through efficient fine-tuning; that is, the augmentation strategy designed based on cloud workload characteristics can maintain important time patterns while increasing data diversity; the multi-scale architecture can simultaneously capture hierarchical time dependencies from hourly to weekly levels; the contrastive learning framework learns robust representations in an unsupervised environment; the efficient fine-tuning strategy prevents catastrophic forgetting and quickly adapts to the target task; combined with cross-cloud environment transfer learning capabilities, the model can be effectively reused across different cloud platforms, ultimately significantly reducing label dependencies, improving prediction accuracy, and lowering deployment costs, providing a practical prediction solution for cloud computing resource management.

[0259] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A self-supervised multi-scale cloud workload prediction method, characterized in that: Includes the following steps: Step S1: Collect time-series data of the workload of each physical machine in the cloud environment. The time-series data includes at least CPU utilization, memory usage, network bandwidth usage, and disk I / O speed. Step S2: Perform data augmentation operations on each of the time series data, including at least multi-scale time warping, frequency domain mixing, and periodic pattern injection, to obtain augmented sequence data; Step S3: Based on the multi-scale patch embedding layer and the scale-aware layer, construct a multi-scale encoder for mapping the enhanced sequence data into a multi-scale temporal representation; Step S4: Using a hierarchical contrastive learning framework, a multi-scale encoder is trained under self-supervised conditions based on the enhanced sequence data. During the training process, multi-scale contrastive loss and cross-scale consistency loss are combined. Step S5: Add a prediction head to the multi-scale encoder after self-supervised training, introduce an adapter of a specific scale, obtain a preset number of labeled data, fine-tune the adapter parameters of the prediction head and the adapter based on the labeled data, and then construct a cloud workload prediction model. Step S6: Perform workload prediction based on the cloud workload prediction model; In step S2, the multi-scale time warp specifically refers to: The time series data is decomposed using discrete wavelet transform. K Each scale component: ; Where x represents time series data; Indicates the first K The scale component, i.e., the first scale component K Detail coefficients at each scale; Represents the approximation coefficient at the coarsest scale; k Indicates scale level; Distort each of the aforementioned scale components: ; in, Indicates the distorted ; This represents the torsion intensity control parameter, used to ensure that finer scales accept less torsion in order to maintain short-term patterns; , Indicates the basic torsional strength; Represents the twist function; Based on the above as well as Construct time series data with multi-scale time warping: ; in, Represents time-series data after multi-scale time warping; The frequency domain mixing specifically refers to: Acquire first time series data and second time series data, and perform Fourier transform on the first time series data and second time series data to obtain a first frequency domain representation and a second frequency domain representation: ； ； in, This represents the first time series data; This represents the second time series data; The first frequency domain representation is shown; This represents the second frequency domain representation; Show the Fourier transform operator; Identify the set of periodic frequencies of the workloads in the first and second time series data: ; in, Represents a set of periodic frequencies; Indicates the daily cycle frequency; Indicates the periodic frequency; Indicates the monthly cycle frequency; Frequency mixing based on a mixing mask function: ; in, Represents a mixing mask function; Indicates frequency; This indicates Bernoulli random sampling; Represents the probability of a random mixture; Performing a spectrum enhancement operation on the first frequency domain representation and the second frequency domain representation yields a mixed frequency domain signal: ; in, Represents a mixed frequency domain signal; This represents element-wise multiplication; Perform an inverse Fourier transform on the mixed frequency domain signal to obtain the time series data after frequency domain mixing; ; in, This represents time series data after frequency domain mixing; Inverse Fourier transform operator; The periodic pattern injection specifically refers to: Based on domain knowledge of workloads in the cloud environment, inject synthesized lifecycle patterns: ; in, express t The periodic pattern of time; t Indicates the position of a time point in time series data; Indicates the number of injected periodic patterns; This represents the amplitude of the i-th periodic pattern; This represents the frequency of the i-th periodic pattern; This represents the phase of the i-th periodic pattern; Indicates the temporal localization window function; Perform data augmentation operations on the time series data based on the aforementioned periodic pattern: ; in, This represents time series data after periodic pattern injection; This represents the injection intensity coefficient for the periodic pattern.

2. The self-supervised multi-scale cloud workload prediction method as described in claim 1, characterized in that: In step S3, the multi-scale patch embedding layer is used to divide the augmented sequence data into patches of different sizes and embed them: ; in, This represents a multi-scale patch embedding function; This represents augmented sequence data; This indicates a multi-scale patch splicing operation; Indicates the first S Scale-based embedding layer; Indicates the first S The size of the patch according to the scale; The scale-aware layer is used to perform cross-scale interactive learning on each patch through several layers of Transformer blocks to obtain a multi-scale time representation.

3. The self-supervised multi-scale cloud workload prediction method as described in claim 1, characterized in that: In step S4, the total loss function of the multi-scale encoder is: ; in, This represents the total loss value of the multi-scale encoder; Indicates multi-scale contrast loss; This represents the cross-scale consistency loss; Indicates the scale number; as well as Both represent loss balance coefficients.

4. The self-supervised multi-scale cloud workload prediction method as described in claim 1, characterized in that: In step S5, the formula for the prediction head is: ; in, Indicates the prediction head; Indicates the prediction head parameters; Represent real numbers; Indicates the scale number; D Indicates dimension; Indicates the predicted time span; The formula for the aggregated multi-scale time representation of the prediction head is: ; in, This indicates the output of the prediction head; Representing scale S Attention weights; Representing scale S Dedicated multilayer sensor; Indicates the first S Scale ; The formula for the adapter is: ; in, Indicates the first S Scale adapter function; Indicates the hidden state of the input adapter; Indicates the activation function; Represents a dimension-reduced matrix; This represents an upgraded matrix.

5. A self-supervised multi-scale cloud workload prediction system, characterized in that: Includes the following modules: The time series data acquisition module is used to collect time series data of the workload of each physical machine in the cloud environment. The time series data includes at least CPU utilization, memory usage, network bandwidth usage, and disk I / O speed. The data augmentation module is used to perform data augmentation operations on each of the time series data, including at least multi-scale time warping, frequency domain mixing, and periodic pattern injection, to obtain augmented sequence data; A multi-scale encoder construction module is used to construct a multi-scale encoder that maps the enhanced sequence data into a multi-scale temporal representation based on a multi-scale patch embedding layer and a scale-aware layer. The self-supervised training module is used to perform self-supervised training of the multi-scale encoder based on the augmented sequence data using a hierarchical contrastive learning framework, combining multi-scale contrastive loss and cross-scale consistency loss during the training process. The fine-tuning module is used to add a prediction head to the multi-scale encoder after self-supervised training, introduce an adapter of a specific scale, obtain a preset number of labeled data, and fine-tune the adapter parameters of the prediction head and the adapter based on the labeled data, thereby constructing a cloud workload prediction model. The workload prediction module is used to predict workloads based on the cloud workload prediction model. In the data augmentation module, the multi-scale time warp specifically refers to: The time series data is decomposed using discrete wavelet transform. K Each scale component: ; Where x represents time series data; Indicates the first K The scale component, i.e., the first scale component K Detail coefficients at each scale; Represents the approximation coefficient at the coarsest scale; k Indicates scale level; Distort each of the aforementioned scale components: ; in, Indicates the distorted ; This represents the torsion intensity control parameter, used to ensure that finer scales accept less torsion in order to maintain short-term patterns; , Indicates the basic torsional strength; Represents the twist function; Based on the above as well as Construct time series data with multi-scale time warping: ; in, Represents time-series data after multi-scale time warping; The frequency domain mixing specifically refers to: Acquire first time series data and second time series data, and perform Fourier transform on the first time series data and second time series data to obtain a first frequency domain representation and a second frequency domain representation: ; ; in, This represents the first time series data; This represents the second time series data; This represents the first frequency domain representation; This represents the second frequency domain representation; Represents the Fourier transform operator; Identify the set of periodic frequencies of the workloads in the first and second time series data: ; in, Represents a set of periodic frequencies; Indicates the daily cycle frequency; Indicates the periodic frequency; Indicates the monthly cycle frequency; Frequency mixing based on a mixing mask function: ; in, Mixing mask function; f Indicates frequency; This indicates Bernoulli random sampling; p Represents the probability of a random mixture; Performing a spectrum enhancement operation on the first frequency domain representation and the second frequency domain representation yields a mixed frequency domain signal: ; in, Represents a mixed frequency domain signal; This represents element-wise multiplication; Perform an inverse Fourier transform on the mixed frequency domain signal to obtain the time series data after frequency domain mixing; ; in, This represents time series data after frequency domain mixing; This represents the inverse Fourier transform operator; The periodic pattern injection specifically refers to: Based on domain knowledge of workloads in the cloud environment, inject synthesized lifecycle patterns: ; in, express t The periodic pattern of time; t Indicates the position of a time point in time series data; Indicates the number of injected periodic patterns; This represents the amplitude of the i-th periodic pattern; This represents the frequency of the i-th periodic pattern; This represents the phase of the i-th periodic pattern; Indicates the temporal localization window function; Perform data augmentation operations on the time series data based on the aforementioned periodic pattern: ; in, This represents time series data after periodic pattern injection; This represents the injection intensity coefficient for the periodic pattern.

6. The self-supervised multi-scale cloud workload prediction system as described in claim 5, characterized in that: In the multi-scale encoder construction module, the multi-scale patch embedding layer is used to divide the augmented sequence data into patches of different sizes and embed them: ; in, This represents a multi-scale patch embedding function; This represents augmented sequence data; This indicates a multi-scale patch splicing operation; Indicates the first S Scale-based embedding layer; Show the first S The size of the patch according to the scale; The scale-aware layer is used to perform cross-scale interactive learning on each patch through several layers of Transformer blocks to obtain a multi-scale time representation.

7. The self-supervised multi-scale cloud workload prediction system as described in claim 5, characterized in that: In the self-supervised training module, the total loss function of the multi-scale encoder is: ; in, This represents the total loss value of the multi-scale encoder; Indicates multi-scale contrast loss; This represents the cross-scale consistency loss; Indicates the scale number; as well as Both represent loss balance coefficients.

8. The self-supervised multi-scale cloud workload prediction system as described in claim 5, characterized in that: In the fine-tuning module, the formula for the prediction head is: ; in, Indicates the prediction head; Indicates the prediction head parameters; Represent real numbers; S Indicates the scale number; D Indicates dimension; Indicates the predicted time span; The formula for the aggregated multi-scale time representation of the prediction head is: ; in, This indicates the output of the prediction head; Representing scale S Attention weights; Representing scale S Dedicated multilayer sensor; Indicates the first S Scale ; The formula for the adapter is: ; in, Indicates the first S Scale adapter function; Indicates the hidden state of the input adapter; Indicates the activation function; Represents a dimension-reduced matrix; This represents an upgraded matrix.

Images