Machine room load determination method and device, electronic equipment and nonvolatile storage medium

Abstract

The invention discloses a machine room load determination method and device, electronic equipment and a nonvolatile storage medium. The method comprises the following steps: acquiring a target load data sequence corresponding to a target machine room in a first time period; performing convolution operation on the target load data sequence to obtain a query vector, a key vector and a value vector; carrying out attention operation on the query vector, the key vector and the value vector by adopting a zoom dot product attention mechanism to obtain a target data feature; and determining a load prediction result according to the target data features. According to the method and the device, the technical problem of poor machine room load prediction precision caused by difficulty in accurately capturing data volatility when complicated time sequence data such as machine room server working load is processed in related technologies is solved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to a method, apparatus, electronic device, and non-volatile storage medium for determining data center load. Background Technology

[0002] With the rapid development of information technology, telecommunications operators' data centers, as the infrastructure of communication networks, bear an ever-increasing network load and data processing demand. Servers, as key equipment within these data centers, require accurate workload prediction for resource planning, energy management, and maintenance scheduling. However, current technologies for predicting server workload in data centers suffer from insufficient handling of data non-stationarity, long-distance dependencies, periodicity, and sudden fluctuations. This makes it difficult to accurately capture data volatility and time dependence, resulting in low prediction accuracy and limiting the effective scheduling of data center resources and the optimization of energy management.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and non-volatile storage medium for determining data center load, in order to at least solve the technical problem of poor data center load prediction accuracy caused by the difficulty of accurately capturing data volatility when processing complex time-series data such as data center server workload.

[0005] According to one aspect of the embodiments of this application, a method for determining data center load is provided, comprising: acquiring a target load data sequence corresponding to a target data center in a first time period, wherein the target load data sequence is used to characterize the workload change state of the target data center in the first time period; performing a convolution operation on the target load data sequence to obtain a query vector, a key vector, and a value vector, wherein the convolution operation is used to capture local temporal dependencies in the target load data sequence; employing a scaled dot product attention mechanism to perform attention operations on the query vector, the key vector, and the value vector to obtain target data features, wherein the scaled dot product attention mechanism is used to capture long-distance dependencies in the target load data sequence; and determining a load prediction result based on the target data features, wherein the load prediction result is used to characterize the workload state of the target data center in a second time period, the second time period being a time period following the first time period.

[0006] Optionally, obtaining the target load data sequence corresponding to the target data center in the first time period includes: collecting the workload data of the target data center in the first time period, and organizing the collected workload data into a time series format to obtain the original load data sequence; checking whether there are missing values ​​in the original load data sequence, and if there are missing values, calculating the mean of the load index at the time points adjacent to the missing value position in the original load data sequence, and filling the missing values ​​with the mean; and normalizing the original load data sequence after filling the missing values ​​to obtain the target load data sequence.

[0007] Optionally, performing convolution operations on the target load data sequence to obtain query vectors, key vectors, and value vectors includes: using a sliding time window to normalize the target load data sequence in the time dimension to obtain a normalized matrix, wherein the normalization process is used to eliminate the non-stationarity of the data; inputting the normalized matrix into the embedding layer of the target load prediction model for processing to obtain an embedding vector, wherein the target load prediction model is used to predict the workload status of the data center in the future time period, and the embedding layer is used to map the normalized data to a high-dimensional space; and using a one-dimensional convolutional layer in the convolutional coding network of the target load prediction model to perform convolution operations on the embedding vector to obtain query vectors, key vectors, and value vectors.

[0008] Optionally, a scaled dot product attention mechanism is employed to perform attention operations on the query vector, key vector, and value vector to obtain target data features. This includes: using a convolutional coding network, calculating the attention output matrix corresponding to each head based on the query vector, key vector, and value vector, where each head corresponds to an independent scaled dot product attention mechanism; concatenating the attention output matrices corresponding to multiple heads, and performing a linear transformation on the concatenated result to obtain a multi-head representation matrix; performing a residual connection between the multi-head representation matrix and the embedding vector, and performing layer normalization on the result of the residual connection to obtain a layer normalized matrix; inputting the layer normalized matrix into a feedforward neural network in the convolutional coding network for processing to obtain the representation matrix of the feedforward neural network, and performing residual connection and layer normalization again on the representation matrix of the feedforward neural network and the layer normalized matrix to obtain the target data features.

[0009] Optionally, calculating the attention output matrix corresponding to each head based on the query vector, key vector, and value vector includes: calculating the dot product of the query vector and key vector, and scaling the dot product according to the length of the key vector; using a first activation function to convert the scaled dot product into a probability distribution vector, where the probability distribution vector is used to characterize the correlation between different time points; and using the probability distribution vector as attention weights to perform a weighted summation of the value vectors to obtain the attention output matrix.

[0010] Optionally, determining the load prediction result based on the target data features includes: inputting the target data features into the first fully connected layer of the multilayer perceptron decoding network of the target load prediction model for processing to obtain a first output matrix; applying a second activation function to the first output matrix nonlinearly to obtain a nonlinear output matrix, wherein the second activation function is used to introduce nonlinear features and avoid the gradient vanishing problem, and the second activation function includes at least one of the following: GeLU activation function; applying a second fully connected layer of the multilayer perceptron decoding network to process the nonlinear output matrix to obtain a second output matrix, and mapping the second output matrix to the initial prediction result; and performing inverse normalization on the initial prediction result to obtain the final load prediction result.

[0011] Optionally, the training steps of the target load prediction model include: acquiring historical workload data of the target data center and determining a training dataset based on the historical workload data; inputting sample data from the training dataset into the initial load prediction model for processing to obtain the load prediction result output by the initial load prediction model, wherein the initial load prediction model includes a convolutional coding network and a multilayer perceptron decoding network; determining the loss function value based on the load prediction result and the real load data of the training dataset, and adjusting the model parameters of the initial load prediction model through backpropagation based on the loss function value to obtain the target load prediction model.

[0012] According to another aspect of the embodiments of this application, a data center load determination device is also provided, comprising: a data acquisition module, configured to acquire a target load data sequence corresponding to a target data center in a first time period, wherein the target load data sequence is used to characterize the workload change state of the target data center in the first time period; a convolution processing module, configured to perform a convolution operation on the target load data sequence to obtain a query vector, a key vector, and a value vector, wherein the convolution operation is used to capture local temporal dependencies in the target load data sequence; a feature extraction module, configured to perform attention operations on the query vector, the key vector, and the value vector using a scaled dot product attention mechanism to obtain target data features, wherein the scaled dot product attention mechanism is used to capture long-distance dependencies in the target load data sequence; and a load prediction module, configured to determine a load prediction result based on the target data features, wherein the load prediction result is used to characterize the workload state of the target data center in a second time period, the second time period being a time period following the first time period.

[0013] According to another aspect of the embodiments of this application, an electronic device is also provided, including: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the program executes a data center load determination method during runtime.

[0014] According to another aspect of the embodiments of this application, a non-volatile storage medium is also provided, the non-volatile storage medium including a stored computer program, wherein the device where the non-volatile storage medium is located executes a data center load determination method by running the computer program.

[0015] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of a data center load determination method.

[0016] In this embodiment, the target load data sequence corresponding to the target data center in the first time period is obtained, where the target load data sequence is used to characterize the workload change state of the target data center in the first time period. A convolution operation is performed on the target load data sequence to obtain query vectors, key vectors, and value vectors, where the convolution operation is used to capture local temporal dependencies in the target load data sequence. A scaling dot product attention mechanism is used to perform attention operations on the query vectors, key vectors, and value vectors to obtain target data features, where the scaling dot product attention mechanism is used to capture long-distance dependencies in the target load data sequence. Based on the target data features, a load prediction result is determined, where the load prediction result characterizes the workload state of the target data center in the second time period, which is the time period following the first time period. By combining the attention mechanism in Transformer with one-dimensional convolution, the temporal nature of the load data is considered while capturing long-distance dependencies, improving temporal dependence and achieving the purpose of deep analysis of the load data sequence. This effectively improves the adaptability and accuracy of the prediction model, thereby solving the technical problem of poor data center load prediction accuracy caused by the difficulty of accurately capturing data fluctuations when processing complex time-series data such as data center server workload. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a schematic diagram of a method for determining data center load according to an embodiment of this application;

[0019] Figure 2 This is a schematic diagram of the process of a data center workload prediction method based on a CTransformer network according to an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of the architecture of a CTransformer prediction model provided according to an embodiment of this application;

[0021] Figure 4 This is a comparison chart of the prediction effects of different models provided in the embodiments of this application;

[0022] Figure 5 This is a radar chart of different prediction models on two different evaluation metrics according to embodiments of this application;

[0023] Figure 6 This is a schematic diagram of a data center load determination device according to an embodiment of this application;

[0024] Figure 7 This is a hardware structure block diagram of a computer terminal (or electronic device) for implementing a method for determining the load of a computer room, according to an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] To facilitate a better understanding of the embodiments of this application by those skilled in the art, some technical terms or nouns involved in the embodiments of this application are explained as follows:

[0028] Time series: refers to a sequence of values ​​of the same statistical indicator arranged in chronological order of their occurrence. Its core characteristic is that the data points are closely related to time, and there is usually a certain dependency between adjacent data points (such as trends, periodicity, etc.).

[0029] Normalization is a key technique in data preprocessing. It refers to transforming data to a specific range (usually [0, 1] or [-1, 1]) according to certain rules to eliminate the influence of differences in units and magnitudes between different features and make the data comparable.

[0030] Softmax is an activation function commonly used in multi-class classification tasks. It can transform a K-dimensional vector containing arbitrary real numbers into a K-dimensional probability distribution vector, where each element takes values ​​between (0, 1) and the sum of all elements is 1, which conforms to the basic definition of probability.

[0031] GeLU (Gaussian Error Linear Unit) is a smooth activation function. Its core idea is to combine the input value with the cumulative distribution function (CDF) of a Gaussian distribution to achieve a non-linear transformation of the input. GeLU is widely used in the Transformer architecture and has become the mainstream activation function in the field of natural language processing.

[0032] With the rapid development of technologies such as 5G and cloud computing, the scale and equipment density of telecom operators' data centers, serving as the "nerve center" supporting the national communication network, are constantly expanding, leading to a sharp increase in energy consumption. Electricity costs have become a significant expense for operators. Servers account for a large proportion of the energy consumption in data centers. By predicting server workload, it is possible to know the load trends in the future, thus providing decision support for operations and maintenance personnel. Accurately predicting server workload is crucial for achieving energy conservation and consumption reduction in data centers.

[0033] As the core hub of communication networks, data centers experience server workloads far more complex than those in ordinary internet scenarios. Traditional server energy management relies primarily on simple statistical models and manual experience. However, these methods have limitations when dealing with complex time-series data such as server workloads. Traditional statistical analysis methods require high data stationarity, while server workloads often exhibit strong nonlinearity, non-stationarity, and complex periodicity, significantly impacting the prediction accuracy of statistical analysis methods and making it difficult to accurately capture dynamic trends in workload changes. While some methods based on recurrent neural networks (RNNs) and their variants (such as LSTM and GRU) offer advantages in processing time-series data and can capture temporal dependencies, they suffer from vanishing or exploding gradients, performing poorly when handling long-term dependencies. Therefore, for time-series data like data center server workloads, which contain periodicity and sudden fluctuations, recurrent neural network models struggle to fully extract deep temporal correlations, resulting in insufficient accuracy and stability in predictions.

[0034] To address the aforementioned issues, this application provides a solution that combines the attention mechanism in Transformer with one-dimensional convolution. This captures long-distance dependencies while also considering the temporal nature of the load data, thus improving temporal dependence. Normalization is incorporated to address the inherent non-stationarity of the data. Finally, a fully connected layer and GeLU activation function are introduced to prevent error accumulation, effectively mitigating the problem of poor prediction accuracy in existing prediction models due to their difficulty in accurately capturing data volatility. A detailed explanation follows.

[0035] According to an embodiment of this application, a method for determining data center load is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0036] Figure 1 This is a schematic diagram of a method for determining data center load according to an embodiment of this application, as shown below. Figure 1 As shown, the method includes the following steps:

[0037] Step S102: Obtain the target load data sequence corresponding to the target data center in the first time period, wherein the target load data sequence is used to characterize the workload change status of the target data center in the first time period.

[0038] Step S104: Perform a convolution operation on the target payload data sequence to obtain a query vector, a key vector, and a value vector. The convolution operation is used to capture local temporal dependencies in the target payload data sequence.

[0039] Step S106: The scaling dot product attention mechanism is used to perform attention operations on the query vector, key vector and value vector to obtain the target data features. The scaling dot product attention mechanism is used to capture long-distance dependencies in the target load data sequence.

[0040] Step S108: Determine the load prediction result based on the target data characteristics. The load prediction result is used to characterize the workload status of the target data center in the second time period, which is the time period following the first time period.

[0041] By combining the attention mechanism in Transformer with one-dimensional convolution through the above steps, the system can capture long-distance dependencies while also taking into account the temporal nature of load data, thereby improving the temporal dependence and achieving the goal of deep analysis of load data sequences. This effectively improves the adaptability and accuracy of the prediction model, and solves the technical problem of poor data center load prediction accuracy caused by the difficulty of accurately capturing data volatility when dealing with complex time series data such as data center server workloads.

[0042] The method for determining the data center load in steps S102 to S108 of the embodiments of this application will be further described below.

[0043] Figure 2 This is a schematic diagram of a data center workload prediction method based on a CTransformer network according to an embodiment of this application, as shown below. Figure 2 As shown in the figure, this embodiment proposes a data center workload prediction method based on CTransformer networks, which can better capture the nonlinear and fluctuating trends of data center server load data and effectively improve prediction accuracy. The method includes the following steps: First, data is collected to obtain historical workload data of the data center and preprocessed. The obtained dataset is divided into training set, validation set, and test set. At the same time, a load prediction model based on CTransformer networks can be constructed. Then, the initial load prediction model (CTransformer network) is trained using the dataset to obtain a trained data center workload prediction model based on CTransformer networks (i.e., the target load prediction model). After that, the real-time collected data center server load data sequence can be input into the trained CTransformer prediction model (i.e., the target load prediction model) for prediction to obtain the load prediction result.

[0044] The specific training steps for the load prediction model (CTransformer network) are as follows.

[0045] In some embodiments of this application, the training steps of the target load prediction model include: acquiring historical workload data of the target data center and determining a training dataset based on the historical workload data; inputting sample data from the training dataset into an initial load prediction model for processing to obtain the load prediction result output by the initial load prediction model, wherein the initial load prediction model includes a convolutional coding network and a multilayer perceptron decoding network; determining a loss function value based on the load prediction result and the real load data of the training dataset, and adjusting the model parameters of the initial load prediction model through backpropagation based on the loss function value to obtain the target load prediction model.

[0046] Specifically, in this implementation, the initial load prediction model (CTransformer network) can be iteratively trained using a training dataset. During training, sample data from the training dataset is first input into the initial load prediction model for processing. This model includes a convolutional coding network and a multilayer perceptron decoding network. The convolutional coding network, through the fusion of one-dimensional convolution and attention mechanisms, not only captures local temporal dependencies but also long-distance dependencies, effectively improving the extraction efficiency of time-series features. The multilayer perceptron decoding network adopts a multilayer perceptron structure, including fully connected layers and the GeLU activation function, avoiding error accumulation, accelerating inference speed, and giving the model stronger non-linear learning capabilities. During training, the model outputs prediction results for historical load data. Next, the loss function value between the prediction results and the actual load data in the training set is calculated. Based on the loss function value, the model parameters are adjusted using the backpropagation algorithm, continuously optimizing the model until the preset convergence condition or performance metric is reached, thus obtaining the target load prediction model. Meanwhile, a validation set can be introduced to verify the model's generalization ability in real time during the training process, and hyperparameters can be dynamically adjusted to avoid overfitting. A test set can be used to objectively evaluate the final performance of the model, and finally a well-trained data center workload prediction model (i.e., target load prediction model) based on the CTransformer network can be obtained.

[0047] The following section provides a detailed description of the process of load prediction using the target load prediction model in the embodiments of this application, as well as the architecture and data processing flow of the target load prediction model.

[0048] First, obtain the workload data of the target data center in the first time period and preprocess it to obtain the target workload data sequence. The specific steps are as follows.

[0049] In some embodiments of this application, obtaining the target load data sequence corresponding to the target data center in the first time period includes the following steps: collecting the workload data of the target data center in the first time period, and organizing the collected workload data into a time series format to obtain the original load data sequence; checking whether there are missing values ​​in the original load data sequence, and if there are missing values, calculating the mean of the load index at the time points adjacent to the position of the missing value in the original load data sequence, and filling the missing values ​​with the mean; and normalizing the original load data sequence after filling the missing values ​​to obtain the target load data sequence.

[0050] Specifically, to obtain complete load data information for the target data center, workload data for the target data center within a specific time period (e.g., the last 24 hours) can be collected first, and this data can be organized into a time series format to form the original load data sequence. Subsequently, a data integrity check is performed to identify and handle missing values ​​in the sequence. Specifically, if missing values ​​are detected in the sequence, the mean of the load data at adjacent time points is calculated, and this mean is used to fill in the missing values ​​to maintain data continuity. Furthermore, the filled original load data sequence can be normalized to convert the data to a uniform scale, resulting in the target load data sequence.

[0051] Normalization overcomes the impact of differences in the units and magnitudes of the original data, ensuring data comparability and model training stability. These preprocessing steps not only enhance the integrity and consistency of the workload data but also effectively eliminate the interference of data distribution differences on model training. This provides a stable data foundation for feature learning in subsequent network layers, avoiding model convergence difficulties or prediction biases caused by non-stationarity, and effectively improving the prediction accuracy and generalization ability of the data center workload prediction method based on the CTransformer network.

[0052] After obtaining the target load data sequence, it can be input into the target load prediction model for analysis and processing, such as... Figure 3 As shown in the embodiments of this application, the target load prediction model mainly includes two parts: a convolutional coding network and a multilayer perceptron decoding network. When analyzing and processing the target load data sequence, a query vector can first be generated by one-dimensional convolution through the convolutional coding network. ), key vector ( ) and value vector ( Then, the attention mechanism and the feedforward neural network are used to form an encoder to extract features. The proposed multilayer perceptron decoding network uses a fully connected layer and the GeLU activation function for decoding. Finally, the prediction result of server load is obtained. The details are introduced below.

[0053] First, the convolutional coding network generates queries by using the attention mechanism in the Transformer network through convolution operations. ),key( ) and value ( ), as detailed below.

[0054] In some embodiments of this application, performing convolution operations on the target load data sequence to obtain query vectors, key vectors, and value vectors includes: using a sliding time window to normalize the target load data sequence in the time dimension to obtain a normalized matrix, wherein the normalization process is used to eliminate the non-stationarity of the data; inputting the normalized matrix into the embedding layer of the target load prediction model for processing to obtain an embedding vector, wherein the target load prediction model is used to predict the workload status of the data center in the future time period, and the embedding layer is used to map the normalized data to a high-dimensional space; and using a one-dimensional convolutional layer in the convolutional coding network of the target load prediction model to perform convolution operations on the embedding vector to obtain query vectors, key vectors, and value vectors.

[0055] To reduce the impact of non-stationarity of the input sequence, embodiments of this application may first process the input target load data sequence. The time dimension is normalized using the sliding window method to obtain the normalized matrix. Then the normalized matrix The embedding layer of the input target load prediction model yields an embedding vector E. Then, in the attention mechanism layer of the convolutional coding network, the resulting embedding vector E is passed through three one-dimensional convolutional layers with kernel size 3 to generate a query vector Q, a key vector K, and a value vector V. The convolution operation captures local temporal dependencies in the input sequence, while the attention mechanism captures long-range local dependencies between sequences, thus aiding in capturing temporal dependencies. The specific formula is shown below:

[0056]

[0057]

[0058]

[0059] in, X represents the normalized matrix, and X represents the input data (i.e., the target load data sequence). This represents the mean. Let E represent the standard deviation, Embedding represent the embedding layer, and E represent the embedding vector. , , This represents the convolution operation, Q represents the query vector, K represents the key vector, and V represents the value vector.

[0060] This series of steps not only effectively captures the local temporal dependencies of the target sequence, but also, through the attention mechanism, extracts long-distance dependencies across sequence lengths, significantly improving the model's ability to understand complex periodic and nonlinear data.

[0061] Next, the query vector Q, key vector K, and value vector V obtained above can be used to calculate the output matrix through a scaling dot product attention mechanism. The specific steps are as follows.

[0062] In some embodiments of this application, a scaled dot product attention mechanism is used to perform attention operations on the query vector, key vector, and value vector to obtain target data features. This includes the following steps: using a convolutional coding network, based on the query vector, key vector, and value vector, calculates the attention output matrix corresponding to each head, where each head corresponds to an independent scaled dot product attention mechanism; concatenates the attention output matrices corresponding to multiple heads, and performs a linear transformation on the concatenated result to obtain a multi-head representation matrix; performs a residual connection between the multi-head representation matrix and the embedding vector, and performs layer normalization on the result of the residual connection to obtain a layer-normalized matrix; inputs the layer-normalized matrix into a feedforward neural network in the convolutional coding network for processing to obtain the feedforward neural network representation matrix, and performs residual connection and layer normalization again on the feedforward neural network representation matrix and the layer-normalized matrix to obtain the target data features.

[0063] Specifically, by scaling dot product attention, based on the query vector, key vector, and value vector, the attention output matrix corresponding to each head is calculated. Each scaled dot product attention represents a head. The calculation steps of the attention output matrix are as follows.

[0064] In some embodiments of this application, calculating the attention output matrix corresponding to each head based on the query vector, key vector, and value vector includes the following steps: calculating the dot product of the query vector and the key vector, and scaling the dot product according to the length of the key vector; using a first activation function to convert the scaled dot product into a probability distribution vector, wherein the probability distribution vector is used to characterize the correlation between different time points; using the probability distribution vector as attention weights to perform a weighted summation of the value vectors to obtain the attention output matrix.

[0065] Specifically, first, the dot product of the query vector and the key vector is calculated, and then the length of the key vector is determined. Scaling the dot product result helps control the range of attention weights, preventing excessively large values ​​from affecting subsequent calculations. Then, the first activation function, Softmax, is used to convert the scaled dot product into a probability distribution vector. This probability distribution vector reflects the correlation between different time points, providing a quantitative basis for the core of the attention mechanism. Finally, using the obtained probability distribution vector as attention weights, a weighted sum of the value vectors is performed to obtain the attention output matrix. The specific formula is as follows:

[0066]

[0067] in, Let Att() represent the output matrix of the i-th head, and let Att() represent the attention operation. , , Let represent the query vector, key vector, and value vector of the i-th head, respectively, and Softamx represent the first activation function. Indicates the length of the key.

[0068] Next, the attention output matrices of multiple heads are concatenated and linearly transformed to obtain the representation matrix of the multiple heads, namely the multi-head representation matrix H mentioned above. The specific formula is as follows:

[0069]

[0070] Where H represents a multi-head representation matrix, and Concat represents the concatenation operation of the multi-head representations. This represents the learnable parameters.

[0071] Subsequently, the multi-head representation matrix H obtained above can be residually joined with the embedding vector E and then subjected to layer normalization to obtain the layer-normalized matrix. The specific formula is as follows:

[0072]

[0073]

[0074] Where C represents the matrix after residual connection, E represents the embedding vector, H is the multi-head representation matrix, and LayerNorm represents the layer normalization process. The matrix representing the normalization of the layer.

[0075] Finally, the layer-normalized matrix obtained above is... The data is fed into a feedforward neural network to obtain its representation matrix. In this embodiment, the feedforward neural network includes two fully connected layers. Then, the representation matrix of the feedforward neural network is subjected to residual connections and layer normalization again to obtain the convolutional coding feature representation matrix L (i.e., the target data features mentioned above). The specific formula is as follows:

[0076]

[0077]

[0078] Where FNN() represents feedforward neural network processing, The matrix after layer normalization is represented. Let W1 and W2 be the representation matrix of the feedforward neural network, respectively, and let b1 and b2 be the weight matrices of the first and second layers, respectively. Let LayerNorm() denotes layer normalization, and L be the target data feature.

[0079] By utilizing the scaled dot product attention mechanism to operate on query vectors, key vectors, and value vectors, the model's ability to capture time-series data is enhanced. Furthermore, by using residual connections and layer normalization, the vanishing gradient problem common in deep networks is avoided, ensuring the stability of model training and the accuracy of prediction results. This provides a powerful technical means for accurate prediction of server load.

[0080] After obtaining the target data features output by the convolutional coding network, the target load prediction model can continue to use the multilayer perceptron decoding network to analyze and process the target data features, thereby obtaining the load prediction results, as follows.

[0081] In some embodiments of this application, determining the load prediction result based on the target data features includes: inputting the target data features into the first fully connected layer of the multilayer perceptron decoding network of the target load prediction model for processing to obtain a first output matrix; applying a second activation function to the first output matrix for nonlinear processing to obtain a nonlinear output matrix, wherein the second activation function is used to introduce nonlinear features and avoid the gradient vanishing problem, and the second activation function includes at least one of the following: the GeLU activation function; applying a second fully connected layer of the multilayer perceptron decoding network to process the nonlinear output matrix to obtain a second output matrix, and mapping the second output matrix to the initial prediction result; and performing inverse normalization processing on the initial prediction result to obtain the final load prediction result.

[0082] Specifically, the multilayer perceptron decoding network in this embodiment consists of two fully connected layers and a GeLU activation function (second activation function), which replaces the original Transformer decoder. This effectively alleviates the problems of slow inference speed and error accumulation in the original encoder. The GeLU activation function enables the model to learn complex nonlinear relationships. Finally, inverse normalization restores the prediction results to the scale of the original data. The specific steps are as follows:

[0083] The target data features L extracted by the convolutional coding network are input into the first fully connected layer to obtain the first output matrix Z1. Then, the first output matrix Z1 is activated by the GeLU activation function (i.e., the second activation function mentioned above) to obtain a nonlinear output matrix Z2. Finally, the nonlinear output matrix Z2 is processed by the second fully connected layer in the multilayer perceptron decoding network to obtain the second output matrix Dec_out. The second output matrix Dec_out is then mapped to the initial prediction result. Finally, inverse normalization restores the initial prediction result to the scale of the original data, thus obtaining the final load prediction result Output. The specific formula is as follows:

[0084]

[0085]

[0086]

[0087]

[0088]

[0089] Where Z1 represents the output matrix of the first fully connected layer, Z2 represents the non-linear output matrix, GeLU is the activation function, Dec_out represents the output matrix of the second fully connected layer, Linear_layer represents the fully connected layer, L is the convolutional encoded feature representation matrix, and Output is the prediction result. This represents the mean. It represents the standard deviation.

[0090] Through the above processing flow, the CTransformer network introduces a fully connected layer and the GeLU activation function to avoid error accumulation. Compared with traditional activation functions, GeLU has smooth gradient characteristics, which can effectively alleviate the gradient vanishing problem. At the same time, with the reasonable design of the fully connected layer, it reduces the error accumulation in the feature propagation process, thereby solving the problem of insufficient prediction accuracy caused by the inability to capture data volatility in current prediction models. It achieves accurate capture of load data volatility, thus greatly improving prediction accuracy and providing more accurate data support for data center operation and maintenance decisions.

[0091] To verify the effectiveness of the embodiments of this application, this verification example is performed on a CPU load dataset from a company's computer room. The dataset records real load data from January 1, 2011 to December 31, 2011, with a sampling interval of 1 hour and a total of 8760 samples. This dataset converts CPU load into server utilization.

[0092] The above workload dataset is input into the target load prediction model in this embodiment of the application. The evaluation index of the target load prediction model (CTransformer) and other models in this embodiment of the application on the workload dataset is compared through testing. The specific results are shown in the table below.

[0093]

[0094] The evaluation metrics used are mean squared error and mean absolute error. By comparing the errors with other advanced prediction models, the effectiveness of the target load prediction model of this application in terms of prediction performance is better demonstrated. According to the evaluation results, the embodiment of this application is superior to the comparative model in both mean squared error and mean absolute error.

[0095] Furthermore, such as Figure 4 The image shows a comparison of predictions from different models on this dataset. To maintain fairness in the prediction models, during the testing process, the parameters of other advanced comparative models and the target load prediction model in this embodiment were set uniformly. Historical time-series data at the same time step were input, and the predicted values ​​at the same future time step were predicted. Analysis shows that the target load prediction model in this embodiment outperforms other models in both overall and detailed prediction performance. Figure 5 As shown, radar charts of different prediction models on two different evaluation metrics are presented, which can more intuitively demonstrate the advantages of the target load prediction model proposed in this application in terms of prediction accuracy compared with other models.

[0096] This application proposes a data center workload prediction method based on CTransformer networks. Leveraging the core advantages of the attention mechanism, it effectively addresses the problem of long sequence dependencies. A convolutional coding network is employed to deeply integrate one-dimensional convolution with the attention mechanism, accurately capturing the temporal characteristics of the data and leveraging the calculation of attention weights between different positions in the sequence to deeply mine the complex temporal correlations hidden within the data. Simultaneously, the normalization and denormalization operations introduced in the method specifically alleviate the non-stationarity problem of sensor-acquired data, providing a more stable input foundation for model training. Furthermore, the design employing a multilayer perceptron decoding network effectively avoids error accumulation during feature transfer, further ensuring prediction accuracy.

[0097] Compared with traditional methods and other deep learning algorithms in related technologies, the CTransformer model proposed in this application exhibits higher prediction accuracy when handling server workload prediction tasks with periodic, nonlinear, and fluctuating characteristics. Experimental results confirm that the model proposed in this invention can significantly improve the accuracy of server workload prediction, providing scientific and powerful technical support for data center resource optimization scheduling and energy-saving decisions, and helping data centers achieve intelligent and efficient operation.

[0098] According to an embodiment of this application, an embodiment of a data center load determination device is also provided. Figure 6 This is a schematic diagram of a data center load determination device according to an embodiment of this application. Figure 6 As shown, the device includes:

[0099] The data acquisition module 60 is used to acquire the target load data sequence corresponding to the target data center in the first time period, wherein the target load data sequence is used to characterize the workload change status of the target data center in the first time period.

[0100] The convolution processing module 62 is used to perform convolution operations on the target payload data sequence to obtain query vector, key vector and value vector. The convolution operation is used to capture local time dependencies in the target payload data sequence.

[0101] The feature extraction module 64 is used to perform attention operations on the query vector, key vector and value vector using a scaling dot product attention mechanism to obtain target data features. The scaling dot product attention mechanism is used to capture long-distance dependencies in the target payload data sequence.

[0102] The load prediction module 66 is used to determine the load prediction result based on the target data characteristics. The load prediction result is used to characterize the workload status of the target data center in the second time period, which is the time period after the first time period.

[0103] Optionally, obtaining the target load data sequence corresponding to the target data center in the first time period includes: collecting the workload data of the target data center in the first time period, and organizing the collected workload data into a time series format to obtain the original load data sequence; checking whether there are missing values ​​in the original load data sequence, and if there are missing values, calculating the mean of the load index at the time points adjacent to the missing value position in the original load data sequence, and filling the missing values ​​with the mean; and normalizing the original load data sequence after filling the missing values ​​to obtain the target load data sequence.

[0104] Optionally, performing convolution operations on the target load data sequence to obtain query vectors, key vectors, and value vectors includes: using a sliding time window to normalize the target load data sequence in the time dimension to obtain a normalized matrix, wherein the normalization process is used to eliminate the non-stationarity of the data; inputting the normalized matrix into the embedding layer of the target load prediction model for processing to obtain an embedding vector, wherein the target load prediction model is used to predict the workload status of the data center in the future time period, and the embedding layer is used to map the normalized data to a high-dimensional space; and using a one-dimensional convolutional layer in the convolutional coding network of the target load prediction model to perform convolution operations on the embedding vector to obtain query vectors, key vectors, and value vectors.

[0105] Optionally, a scaled dot product attention mechanism is employed to perform attention operations on the query vector, key vector, and value vector to obtain target data features. This includes: using a convolutional coding network, calculating the attention output matrix corresponding to each head based on the query vector, key vector, and value vector, where each head corresponds to an independent scaled dot product attention mechanism; concatenating the attention output matrices corresponding to multiple heads, and performing a linear transformation on the concatenated result to obtain a multi-head representation matrix; performing a residual connection between the multi-head representation matrix and the embedding vector, and performing layer normalization on the result of the residual connection to obtain a layer normalized matrix; inputting the layer normalized matrix into a feedforward neural network in the convolutional coding network for processing to obtain the representation matrix of the feedforward neural network, and performing residual connection and layer normalization again on the representation matrix of the feedforward neural network and the layer normalized matrix to obtain the target data features.

[0106] Optionally, calculating the attention output matrix corresponding to each head based on the query vector, key vector, and value vector includes: calculating the dot product of the query vector and key vector, and scaling the dot product according to the length of the key vector; using a first activation function to convert the scaled dot product into a probability distribution vector, where the probability distribution vector is used to characterize the correlation between different time points; and using the probability distribution vector as attention weights to perform a weighted summation of the value vectors to obtain the attention output matrix.

[0107] Optionally, determining the load prediction result based on the target data features includes: inputting the target data features into the first fully connected layer of the multilayer perceptron decoding network of the target load prediction model for processing to obtain a first output matrix; applying a second activation function to the first output matrix nonlinearly to obtain a nonlinear output matrix, wherein the second activation function is used to introduce nonlinear features and avoid the gradient vanishing problem, and the second activation function includes at least one of the following: GeLU activation function; applying a second fully connected layer of the multilayer perceptron decoding network to process the nonlinear output matrix to obtain a second output matrix, and mapping the second output matrix to the initial prediction result; and performing inverse normalization on the initial prediction result to obtain the final load prediction result.

[0108] Optionally, the training steps of the target load prediction model include: acquiring historical workload data of the target data center and determining a training dataset based on the historical workload data; inputting sample data from the training dataset into the initial load prediction model for processing to obtain the load prediction result output by the initial load prediction model, wherein the initial load prediction model includes a convolutional coding network and a multilayer perceptron decoding network; determining the loss function value based on the load prediction result and the real load data of the training dataset, and adjusting the model parameters of the initial load prediction model through backpropagation based on the loss function value to obtain the target load prediction model.

[0109] It should be noted that each module in the above-mentioned data center load determination device can be a program module (e.g., a set of program instructions to implement a certain function) or a hardware module. For the latter, it can be manifested in the following forms, but is not limited to them: each of the above modules is manifested as a processor, or the functions of each of the above modules are implemented by a processor.

[0110] It should be noted that the data center load determination device provided in this embodiment can be used to perform... Figure 1 The method for determining the data center load shown above is also applicable to the embodiments of this application, and will not be repeated here.

[0111] This application embodiment also provides a non-volatile storage medium, which includes a stored computer program. The device containing the non-volatile storage medium executes the following data center load determination method by running the computer program: acquiring a target load data sequence corresponding to the target data center in a first time period, wherein the target load data sequence is used to characterize the workload change state of the target data center during the first time period; performing a convolution operation on the target load data sequence to obtain a query vector, a key vector, and a value vector, wherein the convolution operation is used to capture local temporal dependencies in the target load data sequence; employing a scaled dot product attention mechanism to perform attention operations on the query vector, key vector, and value vector to obtain target data features, wherein the scaled dot product attention mechanism is used to capture long-distance dependencies in the target load data sequence; and determining a load prediction result based on the target data features, wherein the load prediction result is used to characterize the workload state of the target data center in a second time period, the second time period being the time period following the first time period.

[0112] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the data center load determination method described in various embodiments of this application: obtaining a target load data sequence corresponding to the target data center in a first time period, wherein the target load data sequence is used to characterize the workload change state of the target data center in the first time period; performing a convolution operation on the target load data sequence to obtain a query vector, a key vector, and a value vector, wherein the convolution operation is used to capture local temporal dependencies in the target load data sequence; employing a scaled dot product attention mechanism to perform attention operations on the query vector, key vector, and value vector to obtain target data features, wherein the scaled dot product attention mechanism is used to capture long-distance dependencies in the target load data sequence; and determining a load prediction result based on the target data features, wherein the load prediction result is used to characterize the workload state of the target data center in a second time period, the second time period being the time period following the first time period.

[0113] According to an embodiment of this application, an embodiment of a computer terminal (or electronic device) is also provided. Figure 7 A hardware block diagram of a computer terminal (or electronic device) for implementing a method for determining the load of a computer room is shown. Figure 7As shown, the computer terminal 70 (or electronic device) may include one or more processors 702 (shown as 702a, 702b, ..., 702n in the figure) 702 (processor 702 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 704 for storing data, and a transmission device 706 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 7 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, the computer terminal 70 may also include... Figure 7 The more or fewer components shown, or having the same Figure 7 The different configurations shown.

[0114] It should be noted that the aforementioned one or more processors 702 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be wholly or partially integrated into any other element within the computer terminal 70 (or electronic device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0115] The memory 704 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the data center load determination method in this embodiment. The processor 702 executes various functional applications and data processing by running the software programs and modules stored in the memory 704, thereby realizing the aforementioned data center load determination method. The memory 704 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 704 may further include memory remotely located relative to the processor 702, and these remote memories can be connected to the computer terminal 70 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0116] The transmission device 706 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 70. In one example, the transmission device 706 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 706 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0117] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 70 (or electronic device).

[0118] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0119] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0120] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0121] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0122] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0123] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0124] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for determining the load of a computer room, characterized in that, include: Obtain the target load data sequence corresponding to the target data center in the first time period, wherein the target load data sequence is used to characterize the workload change status of the target data center in the first time period; A convolution operation is performed on the target load data sequence to obtain a query vector, a key vector, and a value vector, wherein the convolution operation is used to capture local temporal dependencies in the target load data sequence; A scaling dot product attention mechanism is used to perform attention operations on the query vector, the key vector, and the value vector to obtain target data features. The scaling dot product attention mechanism is used to capture long-distance dependencies in the target load data sequence. Based on the target data characteristics, a load prediction result is determined, wherein the load prediction result is used to characterize the workload status of the target data center in a second time period, which is a time period following the first time period.

2. The method for determining the data center load according to claim 1, characterized in that, The target load data sequence for the target data center in the first time period includes: The workload data of the target computer room during the first time period is collected, and the collected workload data is organized into a time series format to obtain the original workload data sequence. Check whether there are missing values ​​in the original load data sequence. If there are missing values, calculate the mean of the load index at the time points adjacent to the position of the missing value in the original load data sequence, and fill the missing value with the mean. The original load data sequence after filling in missing values ​​is normalized to obtain the target load data sequence.

3. The method for determining the data center load according to claim 1, characterized in that, Performing a convolution operation on the target payload data sequence yields a query vector, a key vector, and a value vector, including: A sliding time window is used to normalize the target load data sequence in the time dimension to obtain a normalized matrix, wherein the normalization process is used to eliminate the non-stationarity of the data. The normalized matrix is ​​input into the embedding layer of the target load prediction model for processing to obtain the embedding vector. The target load prediction model is used to predict the workload status of the data center in the future time period, and the embedding layer is used to map the normalized data to a high-dimensional space. The embedding vector is convolved using a one-dimensional convolutional layer in the convolutional coding network of the target load prediction model to obtain the query vector, the key vector, and the value vector.

4. The method for determining the data center load according to claim 3, characterized in that, A scaled dot product attention mechanism is used to perform attention operations on the query vector, the key vector, and the value vector to obtain target data features including: Using the convolutional coding network, based on the query vector, the key vector, and the value vector, the attention output matrix corresponding to each head is calculated, wherein each head corresponds to an independent scaling dot product attention mechanism; The attention output matrices corresponding to multiple heads are concatenated, and the concatenated result is linearly transformed to obtain the multi-head representation matrix; The multi-head representation matrix and the embedding vector are residually joined, and the result after residual joining is subjected to layer normalization to obtain the layer normalized matrix. The normalized matrix is ​​input into the feedforward neural network in the convolutional coding network for processing to obtain the representation matrix of the feedforward neural network. The representation matrix of the feedforward neural network and the normalized matrix are then subjected to residual connection and layer normalization again to obtain the target data features.

5. The method for determining the data center load according to claim 4, characterized in that, Based on the query vector, the key vector, and the value vector, the attention output matrix corresponding to each head is calculated as follows: Calculate the dot product of the query vector and the key vector, and scale the dot product according to the length of the key vector; The scaled dot product is converted into a probability distribution vector using a first activation function, wherein the probability distribution vector is used to characterize the correlation between different time points; The probability distribution vector is used as the attention weight, and the value vector is weighted and summed to obtain the attention output matrix.

6. The method for determining the data center load according to claim 3, characterized in that, Based on the target data characteristics, the load prediction results are determined as follows: The target data features are input into the first fully connected layer of the multilayer perceptron decoding network of the target load prediction model for processing to obtain the first output matrix. The first output matrix is ​​nonlinearly processed using a second activation function to obtain a nonlinear output matrix. The second activation function is used to introduce nonlinear features and avoid the gradient vanishing problem. The second activation function includes at least one of the following: GeLU activation function. The second fully connected layer in the multilayer perceptron decoding network is used to process the nonlinear output matrix to obtain a second output matrix, and the second output matrix is ​​mapped to the initial prediction result. The initial prediction result is denormalized to obtain the final load prediction result.

7. The method for determining the data center load according to any one of claims 3 to 6, characterized in that, The training steps for the target load prediction model include: Obtain historical workload data of the target data center, and determine the training dataset based on the historical workload data; The sample data in the training dataset is input into the initial load prediction model for processing to obtain the load prediction result output by the initial load prediction model. The initial load prediction model includes a convolutional coding network and a multilayer perceptron decoding network. Based on the load prediction results and the actual load data of the training dataset, a loss function value is determined, and the model parameters of the initial load prediction model are adjusted through backpropagation based on the loss function value to obtain the target load prediction model.

8. A device for determining the load of a computer room, characterized in that, include: The data acquisition module is used to acquire the target load data sequence corresponding to the target data center in the first time period, wherein the target load data sequence is used to characterize the workload change status of the target data center in the first time period. The convolution processing module is used to perform convolution operations on the target payload data sequence to obtain a query vector, a key vector, and a value vector, wherein the convolution operation is used to capture local temporal dependencies in the target payload data sequence; The feature extraction module is used to perform attention operations on the query vector, the key vector, and the value vector using a scaled dot product attention mechanism to obtain target data features, wherein the scaled dot product attention mechanism is used to capture long-distance dependencies in the target load data sequence; The load prediction module is used to determine the load prediction result based on the target data characteristics, wherein the load prediction result is used to characterize the workload status of the target data center in a second time period, which is a time period following the first time period.

9. An electronic device, characterized in that, include: A memory and a processor, the processor being configured to run a program stored in the memory, wherein the program, when running, executes the data center load determination method according to any one of claims 1 to 7.

10. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored computer program, wherein the device containing the non-volatile storage medium executes the data center load determination method according to any one of claims 1 to 7 by running the computer program.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the data center load determination method according to any one of claims 1 to 7.

Images