A lightweight cloud server load prediction method based on wavelet decomposition and multi-head external attention mechanism

Abstract

A lightweight cloud server load prediction method based on wavelet decomposition and multi-head external attention mechanism relates to the technical field of cloud server load prediction. It effectively combines wavelet transform technology and Transformer architecture, and can quickly and accurately predict the long-term load of various resources in cloud data centers. The prediction sequence is decomposed into subsequences of different frequencies by using wavelet transform technology, highlighting the trend part of the load, and using high-frequency information to refine the prediction effect, so as to accurately capture the time evolution law of the load. By introducing the external attention mechanism module, the performance of the model is significantly improved. On the basis of maintaining the prediction accuracy, the computational complexity is reduced, thereby optimizing the reasoning efficiency of load prediction. At the same time, the external memory unit can deeply explore and utilize the potential influence between different sequences, further improving the prediction ability of the model.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud server load prediction, and in particular to a lightweight cloud server load prediction method based on wavelet decomposition and multi-head external attention mechanism. Background Art

[0002] Many types of cloud tasks are run in large-scale cloud platforms, and each task requires different resources during execution, such as CPU resources, memory resources, and IO resources. The monitoring system of the cloud platform records the resource usage of each server and stores it in the form of time series as monitoring logs. By analyzing the monitoring logs and understanding the time series characteristics of the load, the trend of server resource usage can be known in advance, so as to more reasonably allocate resources for user tasks. However, the complex time pattern of the load sequence imposes many limitations on the prediction model in capturing the multidimensional characteristics of the load. The traditional Transformer-based time series modeling scheme is accompanied by secondary computational complexity, which affects the rapid response capability of the cloud server in real-time dynamic scenarios. Summary of the invention

[0003] In order to overcome the shortcomings of the above technologies, the present invention provides a cloud server load prediction method that effectively overcomes the coding redundancy problem in the multi-head attention mechanism and improves the prediction efficiency.

[0004] The technical solution adopted by the present invention to overcome the technical problems is:

[0005] A lightweight cloud server load prediction method based on wavelet decomposition and multi-head external attention mechanism includes the following steps:

[0006] a) Obtain the historical load feature vectors of T cloud servers and obtain the historical load sequence X 1:T =[x1,x2,...,x i ,...,x T ], where x i is the i-th historical load characteristic vector;

[0007] b) The historical load sequence X 1:T =[x1,x2,...,x i ,...,x T ] is divided into training set, validation set and test set;

[0008] c) Establishing a WETformer model, which consists of an encoder and a decoder;

[0009] d) Input the first T-λ historical load feature vectors in the training set into the encoder of the WETformer model and output the detailed feature vector

[0010] e) Perform wavelet decomposition on the first T-λ historical load feature vectors in the training set to obtain the high-frequency input components of the m-th layer decoder and low frequency input components

[0011] f) The detail feature vector High frequency input component and low frequency input components Input into the decoder of the WETformer model, and output the predicted output X of the future load of the cloud server out .

[0012] Furthermore, in step a), the historical load feature vectors of T cloud servers are obtained from the cluster-trace-v2018 Alibaba Cloud monitoring database.

[0013] Furthermore, in step b), the historical load sequence X 1:T =[x1,x2,...,x i ,...,x T ] is divided into training set, validation set and test set according to 7:1:2.

[0014] Further, step d) comprises the following steps:

[0015] d-1) The encoder of the WETformer model consists of an EA module, a first WD module, a feedforward neural network, and a second WD module;

[0016] d-2) Through the formula Calculate the historical load sequence after adding position information encoding Where X 1:T-λ is the first T-λ historical load feature vectors in the training set, X 1:T-λ =[x1,x2,...,x T-λ ],U pos is the position information encoding set, U pos ={u1,u2,...,u i ,...,u T-λ},u i is the i-th historical load characteristic vector x i The corresponding position information code;

[0017] d-3) The historical load sequence Input into the encoder's EA module, through the formula The query subspace Q is calculated, where W Q is the weight matrix used for projection, through the formula Calculate the external attention map Where M k is the external key storage unit in the external attention mechanism, T is the transpose, for After passing through the Softmax function and the layer normalization function, the external attention map The external value storage unit M in the external attention mechanism v Perform multiplication operation and output the feature map

[0018] d-4) Through the formula Calculate the characteristic F, through the formula F out =F+ReLU(Fω1+β1)ω2+β2 to calculate the feature map F output by the EA module out , where ReLU(·) is the ReLU activation function, ω1 and ω2 are weight parameters of the linear layer, and β1 and β2 are bias parameters;

[0019] d-5) The feature map F out Input to the first WD module of the encoder, use the wavelet transform function to perform wavelet decomposition, and then use the wavelet reconstruction function to combine the low-frequency coefficients and high-frequency coefficients with the wavelet transform function, and reconstruct a single branch into the low-frequency component of the encoder's n-th layer output and high frequency fusion component N is the total number of encoder layers;

[0020] d-6) Fusion of high frequency components Input to the encoder's feedforward neural network through the formula Calculate the output characteristics of the feedforward neural network Where σ(·) is the Sigmoid activation function, and All are trainable parameters;

[0021] d-7) The characteristics Input to the second WD module of the encoder, use the wavelet transform function to perform wavelet decomposition, and then use the wavelet reconstruction function to combine the low-frequency coefficients and high-frequency coefficients with the wavelet transform function, and reconstruct a single branch into the low-frequency component of the encoder's n-th layer output and high frequency fusion component

[0022] d-8) Through the formula Calculate the detailed feature vector output by the encoder layer N In the formula is the learnable transformation matrix.

[0023] Further, step e) comprises the following steps:

[0024] e-1) The first T-λ historical load feature vectors X in the training set 1:T-λ After wavelet decomposition using the wavelet transform function, the low-frequency coefficients and high-frequency coefficients are combined with the wavelet transform function through the wavelet reconstruction function, and a single branch is reconstructed into the high-frequency component of the mth layer input of the decoder and low frequency components

[0025] e-2) The high frequency components and a placeholder D of length L avg Perform splicing operation to obtain high-frequency input components The low frequency component and a placeholder A of length L avg Perform splicing operation to obtain low-frequency input components

[0026] e-3) Through the formula Calculate the high-frequency input component of the m-th layer decoder with added position information Where U″ pos is the position information encoding set, U″ pos ={u1,u2,...,u i ,...,u T-λ+L};

[0027] e-4) Through the formula Calculate the low-frequency input component of the m-th layer decoder with added position information

[0028] Further, step f) comprises the following steps:

[0029] f-1) The decoder of the WETformer model consists of the first EA module, the first WD module, the second EA module, the second WD module, the feedforward neural network layer, and the third WD module;

[0030] f-2) The high frequency input component Input to the first EA module of the decoder, through the formula Computed query subspace In the formula is the weight matrix used for projection, through the formula Calculate the external attention map In the formula is the external key storage unit in the external attention mechanism, T is the transpose, for After passing through the Softmax function and the layer normalization function, the external attention map External value storage unit in external attention mechanism Perform multiplication operation and output the feature map

[0031] f-3) Through the formula Calculate the feature F de , through the formula Calculate the characteristic graph output by the first EA module In the formula, and are the weight parameters of the linear layer, and All are bias parameters;

[0032] f-4) The feature map Input to the first WD module of the decoder, use the wavelet transform function to perform wavelet decomposition, and then use the wavelet reconstruction function to combine the low-frequency coefficients and high-frequency coefficients with the wavelet transform function, and reconstruct a single branch into the low-frequency component of the m-th layer output of the decoder and high frequency fusion component M is the total number of layers of the decoder;

[0033] f-5) The high frequency fusion component With the detail feature vector Perform the sum operation to obtain the eigenvector The feature vector Input to the second EA module of the decoder, through the formula The calculated query subspace Q′, where W q ′ is the weight matrix used for projection, through the formula Calculate the external attention map Y′, where M′ k is the external key storage unit in the external attention mechanism, T is the transpose, for After the Softmax function and the layer normalization function, the external attention map Y′ is combined with the external value storage unit M′ in the external attention mechanism. v Perform a multiplication operation and output the feature map F′;

[0034] f-6) Through the formula Calculate the feature F″;

[0035] f-7) Input the feature F″ into the second WD module of the decoder, use the wavelet transform function to perform wavelet decomposition, and then combine the low-frequency coefficients and high-frequency coefficients with the wavelet transform function through the wavelet reconstruction function, and reconstruct a single branch into the low-frequency component of the m-th layer output of the decoder and high frequency fusion component

[0036] f-8) The high frequency fusion component Input into the feedforward neural network layer of the decoder, through the formula Calculate the output characteristics of the feedforward neural network W″ and b″ are both trainable parameters;

[0037] f-9) Through the formula The feature D″′ is calculated;

[0038] f-10) Input the feature D″′ into the third WD module of the decoder, use the wavelet transform function to perform wavelet decomposition, and then combine the low-frequency coefficients and high-frequency coefficients with the wavelet transform function through the wavelet reconstruction function, and reconstruct a single branch into the low-frequency component of the m-th layer output of the decoder and high frequency fusion component

[0039] f-11) by formula Calculate the output of the low-frequency branch of the decoder layer m Where W m,1 For projecting low frequency components The weight, W m,2 For projecting low frequency components The weight, W m,3 For projecting low frequency components The weight of

[0040] f-12) through the formula Calculate the predicted output X of the future load of the cloud server out , where W D is the mapping matrix, It is the high-frequency fusion component output by the Mth layer of the decoder of the third WD module.

[0041] The beneficial effects of the present invention are as follows: the WETformer model decomposes the prediction sequence into high-frequency and low-frequency components through wavelet multi-scale transform. The low-frequency component is the main trend and slowly changing part of the sequence, and contains the global characteristics of the sequence. The high-frequency component is the detail and rapidly changing part of the sequence, and contains the local characteristics of the sequence. Through the wavelet decomposition module, the WETformer model can infer the complex time pattern of the load sequence at different frequencies. In addition, inspired by the successful application of external attention in target detection tasks, a lightweight multi-head external attention mechanism is introduced to reduce the computational complexity of the model and improve the information representation ability of the model. The encoder of the model gradually screens out the detail components of the load sequence through alternating decomposition and feature extraction during the prediction process, and serves as the interactive information within the model. The decoder extracts the trend part from the hidden variables, and then fuses the separated detail information with the trend information, so that the WETformer model can accurately grasp the evolution law and correlation relationship of each component in time. The wavelet transform technology and the Transformer architecture are effectively combined to quickly and accurately predict the long-term load of various resources in the cloud data center. Wavelet transform technology is used to decompose the prediction sequence into subsequences of different frequencies, highlighting the trend of the load, and using high-frequency information to refine the prediction effect, so as to accurately capture the time evolution of the load. By introducing an external attention mechanism module, the performance of the model is significantly improved. On the basis of maintaining the prediction accuracy, the computational complexity is reduced, thereby optimizing the reasoning efficiency of load prediction. At the same time, the external memory unit can deeply explore and utilize the potential influence between different sequences, further improving the prediction ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flow chart of the method of the present invention;

[0043] Figure 2 It is a structural diagram of the discrete wavelet of the present invention;

[0044] Figure 3 This is a structural diagram of the multi-head external attention mechanism of the present invention. DETAILED DESCRIPTION

[0045] The following is combined with Figure 1 , Attachment Figure 2 , Attachment Figure 3 The present invention is further described.

[0046] A lightweight cloud server load prediction method based on wavelet decomposition and multi-head external attention mechanism includes the following steps:

[0047] a) Obtain the historical load feature vectors of T cloud servers and obtain the historical load sequence X 1:T =[x1,x2,...,xi ,...,x T ], where x i is the i-th historical load characteristic vector.

[0048] b) The historical load sequence X 1:T =[x1,x2,...,x i ,...,x T ] is divided into training set, validation set and test set.

[0049] c) Establish a WETformer model, which consists of an encoder and a decoder.

[0050] d) Input the first T-λ historical load feature vectors in the training set into the encoder of the WETformer model and output the detailed feature vector

[0051] e) Perform wavelet decomposition on the first T-λ historical load feature vectors in the training set to obtain the high-frequency input components of the m-th layer decoder and low frequency input components

[0052] f) The detail feature vector High frequency input component and low frequency input components Input into the decoder of the WETformer model, and output the predicted output X of the future load of the cloud server out .

[0053] By embedding discrete wavelet transform as an internal decomposition module, long-term trend information is gradually separated from the load sequence, thereby highlighting the inherent properties of the time series. At the same time, a lightweight multi-head external attention mechanism is introduced as a feature extraction module. This component calculates the internal elements of the load sequence and the correlation between different sequences through a linear structure, effectively overcoming the coding redundancy problem in the multi-head attention mechanism and improving the prediction efficiency. The encoder of the model encodes the high-frequency components of the load as embedded information to capture the local characteristics of the load. The embedded features are then input into the decoder as interactive information to explore the correlation between multiple potential factors that affect the load trend, thereby improving the prediction results.

[0054] In one embodiment of the present invention, in step a), the historical load feature vectors of T cloud servers are obtained from the cluster-trace-v2018 Alibaba Cloud monitoring database.

[0055] In one embodiment of the present invention, in step b), the historical load sequence X 1:T =[x1,x2,...,x i,...,x T ] is divided into training set, validation set and test set according to 7:1:2.

[0056] In one embodiment of the present invention, step d) comprises the following steps:

[0057] d-1) The encoder of the WETformer model consists of an EA module, a first WD module, a feedforward neural network, and a second WD module.

[0058] d-2) Through the formula Calculate the historical load sequence after adding position information encoding Where X 1:T-λ is the first T-λ historical load feature vectors in the training set, X 1:T-λ =[x1,x2,...,x T-λ ],U pos is the position information encoding set, U pos ={u1,u2,...,u i ,...,u T-λ},u i is the i-th historical load characteristic vector x i The corresponding position information is encoded. Historical load sequence is the input vector of the encoder containing position information.

[0059] d-3) The historical load sequence Input into the encoder's EA module, through the formula The query subspace Q is calculated, where W Q is the weight matrix used for projection, through the formula Calculate the external attention map Where M k is the external key storage unit in the external attention mechanism, T is the transpose, for After passing through the Softmax function and the layer normalization function, the external attention map The external value storage unit M in the external attention mechanism v Perform multiplication operation and output the feature map

[0060] d-4) Through the formula Calculate the characteristic F, through the formula F out =F+ReLU(Fω1+β1)ω2+β2 to calculate the feature map F output by the EA module out , where ReLU(·) is the ReLU activation function, ω1 and ω2 are the weight parameters of the linear layer, and β1 and β2 are the bias parameters.

[0061] d-5) The feature map F out Input to the first WD module of the encoder, use the wavelet transform function to perform wavelet decomposition, and then use the wavelet reconstruction function to combine the low-frequency coefficients and high-frequency coefficients with the wavelet transform function, and reconstruct a single branch into the low-frequency component of the encoder's n-th layer output and high frequency fusion component N is the total number of encoder layers.

[0062] d-6) Fusion of high frequency components Input to the encoder's feedforward neural network through the formula Calculate the output characteristics of the feedforward neural network Where σ(·) is the Sigmoid activation function, and are all trainable parameters.

[0063] d-7) The characteristics Input to the second WD module of the encoder, use the wavelet transform function to perform wavelet decomposition, and then use the wavelet reconstruction function to combine the low-frequency coefficients and high-frequency coefficients with the wavelet transform function, and reconstruct a single branch into the low-frequency component of the encoder's n-th layer output and high frequency fusion component d-8) Through the formula Calculate the detailed feature vector output by the encoder layer N In the formula is the learnable transformation matrix.

[0064] In one embodiment of the present invention, step e) comprises the following steps:

[0065] e-1) The first T-λ historical load feature vectors X in the training set 1:T-λ After wavelet decomposition using the wavelet transform function, the low-frequency coefficients and high-frequency coefficients are combined with the wavelet transform function through the wavelet reconstruction function, and a single branch is reconstructed into the high-frequency component of the mth layer input of the decoder and low frequency components

[0066] e-2) The high frequency components and a placeholder D of length L avg Perform splicing operation to obtain high-frequency input components The low frequency component and a placeholder A of length L avg Perform splicing operation to obtain low-frequency input components

[0067] e-3) Through the formula Calculate the high-frequency input component of the m-th layer decoder with added position information Where U″ pos is the position information encoding set, U″ pos ={u1,u2,...,u i ,...,u T-λ+L}.

[0068] e-4) Through the formula Calculate the low-frequency input component of the m-th layer decoder with added position information

[0069] In one embodiment of the present invention, step f) comprises the following steps:

[0070] f-1) The decoder of the WETformer model consists of the first EA module, the first WD module, the second EA module, the second WD module, the feedforward neural network layer, and the third WD module.

[0071] f-2) The high frequency input component Input to the first EA module of the decoder, through the formula Computed query subspace In the formula is the weight matrix used for projection, through the formula Calculate the external attention map In the formula is the external key storage unit in the external attention mechanism, T is the transpose, for After passing through the Softmax function and the layer normalization function, the external attention map External value storage unit in external attention mechanism Perform multiplication operation and output the feature map

[0072] f-3) Through the formula Calculate the feature F de , through the formula Calculate the characteristic graph output by the first EA module In the formula, and are the weight parameters of the linear layer, and These are bias parameters.

[0073] f-4) The feature map Input to the first WD module of the decoder, use the wavelet transform function to perform wavelet decomposition, and then use the wavelet reconstruction function to combine the low-frequency coefficients and high-frequency coefficients with the wavelet transform function, and reconstruct a single branch into the low-frequency component of the m-th layer output of the decoder and high frequency fusion component M is the total number of decoder layers.

[0074] f-5) The high frequency fusion component With the detail feature vector Perform the sum operation to obtain the eigenvector The feature vector Input to the second EA module of the decoder, through the formula The calculated query subspace Q′, where W q ′ is the weight matrix used for projection, through the formula Calculate the external attention map Y′, where M′ k is the external key storage unit in the external attention mechanism, T is the transpose, for After the Softmax function and the layer normalization function, the external attention map Y′ is combined with the external value storage unit M in the external attention mechanism. v ’ is multiplied and the output is the feature map F’.

[0075] f-6) Through the formula The feature F″ is calculated.

[0076] f-7) Input the feature F″ into the second WD module of the decoder, use the wavelet transform function to perform wavelet decomposition, and then combine the low-frequency coefficients and high-frequency coefficients with the wavelet transform function through the wavelet reconstruction function, and reconstruct a single branch into the low-frequency component of the m-th layer output of the decoder and high frequency fusion component

[0077] f-8) The high frequency fusion component Input into the feedforward neural network layer of the decoder, through the formula Calculate the output characteristics of the feedforward neural network W″ and b″ are both trainable parameters.

[0078] f-9) Through the formula The feature D″′ is calculated.

[0079] f-10) Input the feature D″′ into the third WD module of the decoder, use the wavelet transform function to perform wavelet decomposition, and then combine the low-frequency coefficients and high-frequency coefficients with the wavelet transform function through the wavelet reconstruction function, and reconstruct a single branch into the low-frequency component of the m-th layer output of the decoder and high frequency fusion component

[0080] f-11) by formula Calculate the output of the low-frequency branch of the decoder layer m Where W m,1 For projecting low frequency components The weight, W m,2 For projecting low frequency components The weight, W m,3 For projecting low frequency components The weight of .

[0081] f-12) through the formula Calculate the predicted output X of the future load of the cloud server out , where is the high-frequency fusion component output by the decoder layer M of the third WD module, W D To transform the high frequency components of the depth The projection matrix projected to the target dimension.

[0082] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A lightweight cloud server load prediction method based on wavelet decomposition and multi-head external attention mechanism, characterized in that: The steps include: a) Obtain the historical load feature vectors of T cloud servers and obtain the historical load sequence X 1:T =[x1,x2,...,x i ,...,x T ], where x i is the i-th historical load characteristic vector; b) The historical load sequence X 1:T =[x1,x2,...,x i ,...,x T ] is divided into training set, validation set and test set; c) Establishing a WETformer model, which consists of an encoder and a decoder; d) Input the first T-λ historical load feature vectors in the training set into the encoder of the WETformer model and output the detailed feature vector e) Perform wavelet decomposition on the first T-λ historical load feature vectors in the training set to obtain the high-frequency input components of the m-th layer decoder and low frequency input components f) The detail feature vector High frequency input component and low frequency input components Input into the decoder of the WETformer model, and output the predicted output X of the future load of the cloud server out ; Step d) comprises the following steps: d-1) The encoder of the WETformer model consists of an EA module, a first WD module, a feedforward neural network, and a second WD module; d-2) Through the formula Calculate the historical load sequence after adding position information encoding Where X 1:T-λ is the first T-λ historical load feature vectors in the training set, X 1:T-λ =[x1,x2,...,x T-λ ],U pos is the position information encoding set, U pos ={u1,u2,...,u i ,...,u T-λ },u i is the i-th historical load characteristic vector x i The corresponding position information code; d-3) The historical load sequence Input into the encoder's EA module, through the formula The query subspace Q is calculated, where W Q is the weight matrix used for projection, through the formula Calculate the external attention map Where M k is the external key storage unit in the external attention mechanism, T is the transpose, for After passing through the Softmax function and the layer normalization function, the external attention map The external value storage unit M in the external attention mechanism v Perform multiplication operation and output the feature map d-4) Through the formula Calculate the characteristic F, through the formula F out =F+ReLU(Fω1+β1)ω2+β2 to calculate the feature map F output by the EA module out , where ReLU(·) is the ReLU activation function, ω1 and ω2 are weight parameters of the linear layer, and β1 and β2 are bias parameters; d-5) The feature map F out Input to the first WD module of the encoder, use the wavelet transform function to perform wavelet decomposition, and then use the wavelet reconstruction function to combine the low-frequency coefficients and high-frequency coefficients with the wavelet transform function, and reconstruct a single branch into the low-frequency component of the encoder's n-th layer output and high frequency fusion component n∈{1,2,...,N}, N is the total number of encoder layers; d-6) Fusion of high frequency components Input to the encoder's feedforward neural network through the formula Calculate the output characteristics of the feedforward neural network Where σ(·) is the Sigmoid activation function, and All are trainable parameters; d-7) The characteristics Input to the second WD module of the encoder, use the wavelet transform function to perform wavelet decomposition, and then use the wavelet reconstruction function to combine the low-frequency coefficients and high-frequency coefficients with the wavelet transform function, and reconstruct a single branch into the low-frequency component of the encoder's n-th layer output and high frequency fusion component d-8) Through the formula Calculate the detailed feature vector output by the encoder layer N In the formula is a learnable transformation matrix; Step e) comprises the following steps: e-1) The first T-λ historical load feature vectors X in the training set 1:T-λ After wavelet decomposition using the wavelet transform function, the low-frequency coefficients and high-frequency coefficients are combined with the wavelet transform function through the wavelet reconstruction function, and a single branch is reconstructed into the high-frequency component of the mth layer input of the decoder and low frequency components e-2) The high frequency components and a placeholder D of length L avg Perform splicing operation to obtain high-frequency input components The low frequency component and a placeholder A of length L avg Perform splicing operation to obtain low-frequency input components e-3) Through the formula Calculate the high-frequency input component of the m-th layer decoder with added position information Where U″ pos is the position information encoding set, U′ p ' os ={u1,u2,...,u i ,...,u T- λ +L }; e-4) Through the formula Calculate the low-frequency input component of the m-th layer decoder with added position information Step f) comprises the following steps: f-1) The decoder of the WETformer model consists of the first EA module, the first WD module, the second EA module, the second WD module, the feedforward neural network layer, and the third WD module; f-2) The high frequency input component Input to the first EA module of the decoder, through the formula Computed query subspace In the formula is the weight matrix used for projection, through the formula Calculate the external attention map In the formula is the external key storage unit in the external attention mechanism, T is the transpose, for After passing through the Softmax function and the layer normalization function, the external attention map External value storage unit in external attention mechanism Perform multiplication operation and output the feature map f-3) Through the formula Calculate the feature F de , through the formula Calculate the characteristic graph output by the first EA module In the formula, and are the weight parameters of the linear layer, and All are bias parameters; f-4) The feature map Input to the first WD module of the decoder, use the wavelet transform function to perform wavelet decomposition, and then use the wavelet reconstruction function to combine the low-frequency coefficients and high-frequency coefficients with the wavelet transform function, and reconstruct a single branch into the low-frequency component of the m-th layer output of the decoder and high frequency fusion component m∈{1,2,...,M}, M is the total number of decoder layers; f-5) The high frequency fusion component With the detail feature vector Perform the sum operation to obtain the eigenvector The feature vector Input to the second EA module of the decoder, through the formula The calculated query subspace Q′, where W q ' is the weight matrix for projection, through the formula Y' = Norm (Q'M' k T ) calculates the external attention map Y′, where M′ k is the external key storage unit in the external attention mechanism, T is the transposition, and Norm(Q′M′ k T ) is Q′M′ k T After the Softmax function and the layer normalization function, the external attention map Y′ is combined with the external value storage unit M in the external attention mechanism. v ′ is multiplied and the output is the feature map F′; f-6) Through the formula Calculate the feature F″; f-7) Input the feature F″ into the second WD module of the decoder, use the wavelet transform function to perform wavelet decomposition, and then combine the low-frequency coefficients and high-frequency coefficients with the wavelet transform function through the wavelet reconstruction function, and reconstruct a single branch into the low-frequency component of the m-th layer output of the decoder and high frequency fusion component f-8) The high frequency fusion component Input into the feedforward neural network layer of the decoder, through the formula Calculate the output characteristics of the feedforward neural network W″ and b″ are both trainable parameters; f-9) Through the formula The feature D″′ is calculated; f-10) Input the feature D″′ into the third WD module of the decoder, use the wavelet transform function to perform wavelet decomposition, and then combine the low-frequency coefficients and high-frequency coefficients with the wavelet transform function through the wavelet reconstruction function, and reconstruct a single branch into the low-frequency component of the m-th layer output of the decoder and high frequency fusion component f-11) by formula Calculate the output of the low-frequency branch of the mth layer of the decoder Where W m,1 For projecting low frequency components The weight, W m,2 For projecting low frequency components The weight, W m,3 For projecting low frequency components The weight of f-12) through the formula Calculate the predicted output X of the future load of the cloud server out , where W D is the mapping matrix, It is the high-frequency fusion component output by the Mth layer of the decoder of the third WD module.

2. The lightweight cloud server load prediction method based on wavelet decomposition and multi-head external attention mechanism according to claim 1 is characterized in that: In step a), the historical load feature vectors of T cloud servers are obtained from the cluster-trace-v2018 Alibaba Cloud monitoring database.

3. The lightweight cloud server load prediction method based on wavelet decomposition and multi-head external attention mechanism according to claim 1 is characterized in that: In step b), the historical load sequence X 1:T =[x1,x2,...,x i ,...,x T ] is divided into training set, validation set and test set according to 7:1:2.

Images