A training method and device of a prediction model, an electronic device, and a storage medium

Abstract

This application discloses a training method, apparatus, electronic device, and storage medium for a prediction model. The method includes: acquiring raw time-series data; preprocessing the raw time-series data to obtain exogenous variables, which represent the extension of periodic data corresponding to the raw time-series data to a first preset time period; inputting the raw time-series data and exogenous variables into a preset model, and processing the raw time-series data based on the exogenous variables and a locality-sensitive hash function to obtain predicted time-series data corresponding to the first preset time period; training the preset model based on the predicted time-series data and the preset time-series data until the preset model meets preset training conditions, thereby obtaining a trained prediction model. According to the embodiments of this application, the problem of low accuracy in data prediction can be effectively solved.

Description

Technical Field

[0001] This application belongs to the field of Internet technology, and in particular relates to a method, apparatus, electronic device and storage medium for training a prediction model. Background Technology

[0002] With the development of information technology, in the field of intelligent operations and maintenance (O&M), analyzing historical indicator data, deciphering its inherent time-series patterns, and predicting future trends are crucial. This type of time-series data typically consists of a timestamp and its corresponding multivariate vector values. Applications of time series forecasting in intelligent O&M can include network throughput, disk utilization, and traffic volume prediction, among others.

[0003] Currently, time series data prediction methods are mainly divided into traditional statistical analysis methods and deep learning-based sequence modeling methods. However, existing time series prediction methods are often based on assumptions about the distribution of large amounts of data and only analyze and capture the linear relationship of time series data, resulting in low accuracy in data prediction. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and storage medium for training a prediction model, which can solve the problem of low accuracy in current data prediction.

[0005] In a first aspect, embodiments of this application provide a method for training a prediction model, the method comprising:

[0006] Obtain raw time series data;

[0007] The original time series data is preprocessed to obtain exogenous variables, which are used to characterize the extension of the periodic data corresponding to the original time series data to a first preset time period.

[0008] The original time series data and exogenous variables are input into the preset model. Based on the exogenous variables and the locality-sensitive hash function, the original time series data is processed to obtain the predicted time series data corresponding to the first preset time period.

[0009] Based on the predicted time series data and the preset time series data, a preset model is trained until the preset model meets the preset training conditions, and a well-trained prediction model is obtained.

[0010] Secondly, embodiments of this application provide a training apparatus for a prediction model, the training apparatus for the prediction model comprising:

[0011] The acquisition module is used to acquire raw time-series data;

[0012] The preprocessing module is used to preprocess the original time series data to obtain exogenous variables. The exogenous variables are used to represent the extension of the periodic data corresponding to the original time series data to a first preset time period.

[0013] The input module is used to input the original time series data and exogenous variables into the preset model, and process the original time series data based on the exogenous variables and the locality-sensitive hash function to obtain the predicted time series data corresponding to the first preset time period.

[0014] The training module is used to train a preset model based on the predicted time series data and the preset time series data until the preset model meets the preset training conditions, thus obtaining a trained prediction model.

[0015] Thirdly, embodiments of this application provide an electronic device, the device including: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the method as described in the first aspect or any possible implementation of the first aspect.

[0016] Fourthly, embodiments of this application provide a readable storage medium storing computer program instructions that, when executed by a processor, implement the method as described in the first aspect or any possible implementation thereof.

[0017] In this embodiment, the original time-series data is preprocessed to obtain exogenous variables that characterize the periodic data corresponding to the original time-series data extended to a first preset time period. Here, the periodic data corresponding to the original time-series data can be incorporated into the exogenous variables, facilitating the integration of domain-specific features from the actual usage scenario into the subsequent prediction process. Then, the original time-series data and exogenous variables are input into a preset model. Based on the exogenous variables and a locality-sensitive hash function, the original time-series data is processed to obtain predicted time-series data corresponding to the first preset time period. Since the exogenous variables are information extracted from the original time-series data, autoregressive inference is unnecessary during model inference, enabling multi-level parallel inference and accelerating the inference speed. Finally, the preset model is trained based on the predicted time-series data and the preset time-series data until the preset model meets preset training conditions, resulting in a trained prediction model. Therefore, the trained prediction model can quickly and accurately predict the predicted time-series data corresponding to the original time-series data. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a training method for a prediction model provided in an embodiment of this application;

[0020] Figure 2 This is a schematic diagram of a preprocessing procedure provided in an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of the structure of a prediction model provided in an embodiment of this application;

[0022] Figure 4 This is a schematic diagram of the structure of a generative adversarial network provided in an embodiment of this application;

[0023] Figure 5 This is a schematic diagram of the structure of a training device for a prediction model provided in an embodiment of this application;

[0024] Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] The features and exemplary embodiments of various aspects of this application will now be described in detail. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only configured to explain this application and are not configured to limit this application. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples of this application.

[0026] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0027] The following is a brief introduction to the technical terms used in this application.

[0028] Throughput originally refers to the output or service volume provided by a business system per unit of time. In computer or data communication systems, it refers to the average rate at which data is successfully delivered through a communication channel or node per unit of time, usually measured in bits per second (bps).

[0029] CPU utilization is essentially the amount of central processing unit (CPU) resources used by running programs, indicating the amount of programs running on the machine at a given point in time. Higher utilization means the machine is running many programs at that time, and vice versa. Utilization is directly related to CPU power.

[0030] Long Short-Term Memory (LSTM) is a type of recurrent neural network designed to address the long-term dependency problem inherent in general recurrent neural networks. All recurrent neural networks have a chain-like structure of repeating neural network modules.

[0031] The Transformer architecture replaces LSTM with a fully attention-based structure, abandoning the traditional encoder-decoder model's inherent requirement to combine it with Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs). It achieves better results while reducing computational cost and improving parallel efficiency. The Transformer model abandons the idea of ​​sequential modeling, using self-attention as the kernel of deep models, thus enabling parallel training and effectively mitigating the error accumulation effect during the training phase of regressive neural networks.

[0032] Self-attention is a key concept in Transformer. As the name suggests, self-attention refers to the attention mechanism that occurs between internal elements of data. Clearly, self-attention is more effective at handling long-range interdependent features. This is because RNNs or LSTMs require calculating the target based on time series data. For long-range interdependent features, it takes several time steps of information accumulation to connect the two, and the greater the distance, the more difficult it is to extract effective information. However, self-attention directly represents the connection between any two targets through a single calculation result, greatly reducing the distance between long-range dependent features and facilitating the effective use of these features.

[0033] Prophet is a method for forecasting time series data based on additive models, where nonlinear trends align with annual, weekly, and daily seasonal variations as well as holiday effects. It is suitable for time series with strong seasonal influences and historical data from multiple seasons.

[0034] Exogenous variables are variables that affect the system but are not affected by it. In econometric models, exogenous variables are variables that are uncorrelated with the model's random disturbance term. Typically, variables describing external factors affecting the operation of an economic system, such as technology, politics, institutions, and natural conditions, are exogenous variables.

[0035] The training method for the prediction model provided in this application embodiment can be applied to at least the following application scenarios, which will be described below.

[0036] With the development of information technology, ensuring the stability of digital systems has become a fundamental requirement for enterprises. In the field of intelligent operations and maintenance, analyzing historical indicator data, deciphering its inherent time-series patterns, and predicting future trends are crucial.

[0037] This type of data typically consists of a timestamp and its corresponding multivariate vector values. Applications of time series forecasting in intelligent operations and maintenance (O&M) can include network throughput prediction, disk utilization prediction, CPU utilization prediction, traffic volume prediction, and virtual system capacity prediction. Time series forecasting technology is a cornerstone of the intelligent O&M technology field.

[0038] Mature time prediction technologies can serve as middleware for other intelligent operations and maintenance technologies, such as anomaly detection and root cause analysis, thereby improving their corresponding performance. Root cause analysis, in particular, is a structured problem-solving method used to progressively identify and resolve the root causes of problems, rather than merely focusing on the symptoms of the problem.

[0039] Existing time series forecasting methods are mainly divided into traditional statistical analysis methods and deep learning-based sequence modeling methods, such as recurrent neural networks, which have emerged in recent years. However, traditional statistical analysis methods are often based on assumptions about the distribution of large amounts of data and only analyze and capture linear relationships in time series data, lacking generality and extensibility. While existing deep learning-based sequence modeling methods can capture complex higher-order patterns, they suffer from the following problems:

[0040] First, the model relies on autoregression during the inference phase, making long-sequence prediction infeasible. Second, the model's time complexity is too high, making large-scale industrial deployment difficult. Third, the model exhibits high instability, especially for modeling sequence regression, such as RNN-type models, where this problem is particularly pronounced. Finally, deep models struggle to encode user domain knowledge, resulting in a lack of real-world experience constraints and making them prone to overfitting.

[0041] Existing deep learning-based sequence modeling methods still suffer from difficulties in parallelizing training and error accumulation, making large-scale industrial deployment a pain point in the industrial technology field.

[0042] Transformer has effectively improved the performance of sequence models and has received widespread research attention in the fields of Neuro-Linguistic Programming (NLP) and image processing. However, for time series data, since the sequences are often much longer than those in NLP tasks, Transformer still suffers from low computational efficiency on long-term sequences. Furthermore, its inference phase is still autoregressive and suffers from error accumulation, which poses a significant challenge to its industrial application.

[0043] Figure 1 This is a flowchart of a training method for a prediction model provided in an embodiment of this application.

[0044] like Figure 1 As shown, the training method for this prediction model may include steps 110-140. This method is applied to the training device for the prediction model, as detailed below:

[0045] Step 110: Obtain the raw time series data.

[0046] Step 120: Preprocess the original time series data to obtain exogenous variables. The exogenous variables are used to characterize the extension of the periodic data corresponding to the original time series data to a first preset time period.

[0047] Step 130: Input the original time series data and exogenous variables into the preset model, and process the original time series data based on the exogenous variables and the locality-sensitive hash function to obtain the predicted time series data corresponding to the first preset time period.

[0048] Step 140: Based on the predicted time series data and the preset time series data, train the preset model until the preset model meets the preset training conditions, and obtain the trained prediction model. The time series information of the predicted time series data matches the time series information of the preset time series data.

[0049] In this embodiment, the original time-series data is preprocessed to obtain exogenous variables that characterize the periodic data corresponding to the original time-series data extended to a first preset time period. Here, the periodic data corresponding to the original time-series data can be incorporated into the exogenous variables, facilitating the integration of domain-specific features from the actual usage scenario into the subsequent prediction process. Then, the original time-series data and exogenous variables are input into a preset model. Based on the exogenous variables and a locality-sensitive hash function, the original time-series data is processed to obtain predicted time-series data corresponding to the first preset time period. Since the exogenous variables are information extracted from the original time-series data, autoregressive inference is unnecessary during model inference, enabling multi-level parallel inference and accelerating the inference speed. Finally, the preset model is trained based on the predicted time-series data and the preset time-series data until the preset model meets preset training conditions, resulting in a trained prediction model. Therefore, the trained prediction model can quickly and accurately predict the predicted time-series data corresponding to the original time-series data.

[0050] The following describes the contents of steps 110-140 respectively:

[0051] Step 110 is involved.

[0052] Obtain the raw time series data.

[0053] The raw time series data is data related to time series, such as network throughput, disk utilization, and CPU utilization corresponding to time series.

[0054] Step 120 is involved.

[0055] The original time series data is preprocessed to obtain exogenous variables, which are used to characterize the extension of the periodic data corresponding to the original time series data to a first preset time period.

[0056] Among them, the original time series data corresponds to the original time period, and the first preset time period is after the original time period.

[0057] By leveraging the versatility of the Prophet model, simple and easy-to-use data preprocessing can be achieved to obtain exogenous variables. These exogenous variables are determined based on the periodic information in the original time series data, thus carrying some holiday information. This allows users to easily understand the periodicity and holiday information and integrate it into the data preprocessing process.

[0058] This application extends the periodic terms extracted by the Prophet algorithm to the prediction length as an exogenous time series. It does not require a masking mechanism. Since the exogenous variables are information extracted from the original time series data, there is no need for autoregressive inference in the inference stage. It can achieve multi-level parallel inference, which greatly accelerates the inference speed. Furthermore, since the model abandons autoregressive inference, the model is no longer troubled by the error accumulation problem.

[0059] In one possible embodiment, step 120 may specifically include the following steps:

[0060] Determine the correlation information of the original time series data. The correlation information includes: the variable point annotation information and holiday information in the original time series data.

[0061] Based on the original time series data and related information, the first time series data is determined;

[0062] The first time series data is input into the time series decomposer to obtain exogenous variables.

[0063] Among them, the change point annotation information is the marker point information obtained by automatically annotating the original time series data, which is used to annotate the jumps and abrupt changes in the original time series data.

[0064] To address the current challenge of encoding domain-specific knowledge into deep models, this paper employs the Prophet algorithm, which decomposes time series data. By integrating practitioners' knowledge of the data itself, it extracts periodic information from time series data and uses this periodic information as an exogenous variable to input into subsequent pre-defined models, enabling deep learning models to learn more accurate periodic information.

[0065] First, raw time-series data is collected, and then the associated information of the raw time-series data is selected. The associated information includes variable point annotations and holiday-related information related to the raw time-series data.

[0066] The time series decomposer takes the timestamps and corresponding values ​​of a known time series as input, and the length of the time series to be predicted as input; it outputs the future trend of the time series. The output of the time series decomposer can provide necessary statistical indicators, including the fitted curve, upper bound, and lower bound.

[0067] Specifically, the step of inputting the first time-series data into the time-series decomposer to obtain exogenous variables may include the following steps:

[0068] The first time series data is input into the time series decomposer to extract the parameters corresponding to the Nth order Fourier series of the correlation information, where N is a positive integer;

[0069] Numerical simulations were performed on the parameters to obtain exogenous variables.

[0070] The first time series data is input into the time series decomposer, and the Fourier series expression of the periodic information of the data is obtained through the Prophet time series decomposition algorithm, that is, the parameters corresponding to its Nth order Fourier series.

[0071] Numerical simulations are performed using the parameters of the Fourier series obtained by the Prophet time series decomposition algorithm, thereby extending the periodic data corresponding to the original time series data to the level to be predicted, i.e., the exogenous variables corresponding to the original time series data.

[0072] Specifically, such as Figure 2 As shown, the input to the Prophet time series decomposer is the original time series data labeled with association information, i.e., the first time series data, y[1:t0]. The output of the time series decomposer is the exogenous variable, i.e., X[t1:t1+tau].

[0073] Among them, let the original time series data A time series with length t0 and dimension d.

[0074] The Prophet algorithm treats each dimension of a d-dimensional time series as a one-dimensional unit and processes them separately, obtaining a decomposition of the time series through a regression over time: y(t) = g(t) + s(t) + h(t) + ∈ t .

[0075] Where g(t), s(t), and h(t) are the trend term, periodic term, and holiday term, respectively, ∈ t It is assumed to be a parameterized Gaussian. After regressing g(t), s(t), and h(t) together, the parameters of the periodic term s(t) are the seasonal information extracted by this method.

[0076] Here, the exogenous variable can be determined based on the periodic term s(t).

[0077] Where g(t) represents the trend term, which indicates the trend of change in the time series on a non-periodic basis;

[0078] s(t) represents the periodic term, or seasonal term, which is generally expressed in weeks or years;

[0079] h(t) represents the holiday item, indicating the impact of potential non-fixed-period holidays in the time series on the predicted value;

[0080] The error term, also known as the residual term, represents the fluctuations that the model did not predict and follows a Gaussian distribution.

[0081] The Prophet algorithm obtains the predicted value of the time series by fitting these terms and then summing them up.

[0082] Specifically, g(t) = (k + a(t)) T δ)t+(m+a(t) T γ). Where k is the growth rate of the linear trend, a(t) is a vector function reflecting the information of trend change points, δ vector reflects the rate adjustment corresponding to each change point, m is an offset parameter, and γ j Set to -S j δ j , here s j The time corresponding to the j-th change point is represented by this setting, which makes g(t) a continuous function.

[0083] Breakpoints can be manually set based on an understanding of the data and prophecy. j Alternatively, a series of potential breakpoints can be given to the algorithm for automatic selection, thereby incorporating professional knowledge.

[0084] To enable the algorithm to automatically select breakpoints, you can set δ... j The model follows a Laplace prior distribution, Laplace(0, v). The hyperparameter v is used to control the flexibility of the model in learning the rate of change. It is important to note in the implementation that prophet is only used as a preprocessing periodic information extractor and does not need to consider the case where the prediction j > T. Specifically, prophet in this application is only used to fit and provide exogenous variables.

[0085] For periodic information

[0086] Here, P represents periodic information provided by the practitioner; for example, when the time series has a periodicity, P = 30. N is a hyperparameter controlling the order of the Fourier series. n and b n These are preset coefficients.

[0087] β = [a1, b1, ..., a n b n ] represents the parameters corresponding to the Fourier series, which can be learned through the fitting of prophet, thus obtaining the periodic function s(t) with respect to time, and thus obtaining the exogenous variable: X = [s(1), s(t0+1), ..., s(t0+τ)], which serves as the input to the decoder of this application.

[0088] In implementation, β follows a Gaussian prior distribution Normal(0, σ). 2 ). Where σ 2 As a hyperparameter controlling the smoothness of β, it can achieve the effect of L2 regularization.

[0089] For modeling holiday information, for each holiday i, let D i Let Z(t) = [I(t∈D1), ..., I(t∈D2)] be the set of times affected by holiday i. l [), where I is the Indicator function, and let h(t) = Z(t) T k, the parameter k represents the impact of holidays on the fit, and similarly k ~ Normal(0, Γ) 2 )

[0090] The Prophet-based preprocessing procedure can learn its corresponding parameters using the Max A Posterior (MAP) distribution. This is achieved by studying the parameters of ∈ t By assuming a Gaussian distribution, the MAP corresponding to this generative model has a closed-form expression, which allows the optimization problem to be solved using various numerical optimization software.

[0091] In Bayesian statistics, the "maximum a posteriori (MAP) estimate" is the mode of the posterior probability distribution. MAP can be used to obtain point estimates of quantities in experimental data that cannot be directly observed. It is closely related to the classical method of maximum likelihood estimation, but it uses an augmented optimization objective to further consider the prior probability distribution of the estimated quantity. Therefore, MAP can be viewed as a regularized maximum likelihood estimation.

[0092] Step 130 is involved.

[0093] The original time series data and exogenous variables are input into the preset model. Based on the exogenous variables and the locality-sensitive hash function, the original time series data is processed to obtain the predicted time series data corresponding to the first preset time period.

[0094] To address the inefficiency of the naive Transformer in multivariate time series prediction, this application employs a multi-way locality-sensitive hash function to accelerate the matrix operations of QK^T, resulting in shorter training and inference times for the model.

[0095] In one possible embodiment, step 130 may specifically include the following steps:

[0096] The original time series data and exogenous variables are input into a preset model, and the original time series data is processed based on the locality-sensitive hash function to obtain the second time series data.

[0097] Global latent features are determined based on exogenous variables and second time-series data;

[0098] Based on global hidden features, exogenous variables, and second time series data, predictive time series data corresponding to the first preset time period are determined.

[0099] Among them, global hidden features are used to characterize the hidden features in the original time series data.

[0100] The preset model includes: a sparse self-attention submodule, a forward-linked network submodule, and a network residual connection structure. Accordingly, step 130 may specifically include the following steps:

[0101] Global latent features are determined based on exogenous variables and second time-series data;

[0102] Based on global latent features, exogenous variables, and second time-series data, the predicted time-series data is determined, including:

[0103] The original time series data is input into the sparse self-attention submodule, and the original time series data is processed based on the locality-sensitive hash function to obtain the second time series data.

[0104] The exogenous variables and the second time-series data are input into the forward linking network submodule to capture the global latent features;

[0105] The predicted time series data is obtained by connecting the global hidden features, the exogenous variables, and the second time series data through the network residual link structure.

[0106] First, before the steps mentioned above, which involve processing the original time-series data based on the locality-sensitive hash function to obtain the second time-series data, the following steps may also be included:

[0107] Determine the query matrix and key matrix based on the exogenous variables;

[0108] Determine the locality-sensitive hash function based on the query matrix and the key matrix.

[0109] A sparse self-attention submodule based on the local sensitive hashing function (LSH). This addresses the limitation of the multi-head dot product self-attention mechanism in the Naive Transformer, which requires computing the product QK of the query matrix Q and the key matrix K. T As the length of the time series increases, such computation gradually becomes inefficient. Among them, QK T This refers to the transpose of the query matrix multiplied by the key matrix.

[0110] The query matrix Q and the key matrix K are determined based on exogenous variables and the first time series data.

[0111] Locality-sensitive hashing (LSH) is a method that can quickly find nearest neighbors in high-dimensional space. An LSH algorithm transforms each vector x into hashh(x), where hashes closer to vector x are more likely to have the same hash value, while those farther away are not.

[0112] To simplify the calculation of dot product self-attention, this application adopts a shared Q and K matrix approach, and applies this approach to each query vector q. i Only for its neighboring key vector k j Calculate the dot product.

[0113] Among them, proximity in high-dimensional space is determined by the locality-sensitive hash function h, when q i With k j q only exists when they belong to the same hash value. i and k j The dot product operation. Hash functions, on the other hand, are based on random projection;

[0114] If q i k j All are d k For a vector of dimension (d), then choose a dimension of (d) k If we have a random matrix R (b / 2), then the local sensitive hash function is defined as h(x) = argmax([xR; -xR]).

[0115] Where [u; v] is defined as the glue of vectors u and v. Where d k d represents the dimension of the original data, b is a preset parameter, and x is a preset coefficient. k It is the data dimension of the original time series data.

[0116] Therefore, the expression for the locality-sensitive hash function can be determined based on the query matrix and the key matrix.

[0117] Secondly, the steps mentioned above for determining global latent features based on exogenous variables and second-time-series data may specifically include the following steps:

[0118] Determine the random matrix based on the data dimensions of the original time series data;

[0119] Based on the random matrix, determine the weight parameter matrix and the bias parameter vector;

[0120] Global latent features are obtained by calculating exogenous variables and second time series data based on the weight parameter matrix and bias parameter vector.

[0121] The forward-linked network submodule is used to capture global latent features of multidimensional time series.

[0122] For the input previously sent to the link network submodule This submodule utilizes the built-in weight parameter matrix. And the deviation parameter vector b∈R 1×n Calculate Output = f(RW + b), where Output is a global latent feature.

[0123] The weight parameter matrix and bias parameter vector mentioned above are determined based on a random matrix.

[0124] The input to the forward link network submodule may include: second time series data obtained by processing the original time series data based on a locality-sensitive hash function, and exogenous variables.

[0125] The function f is the non-linear activation function of the network, and f can be adopted as the ReLU function f(x) = max(0,x).

[0126] That is, when RW+b is greater than 0, the global latent feature is RW+b; when RW+b is less than 0, the global latent feature is 0.

[0127] Finally, the predicted time series data are determined based on global hidden features, exogenous variables, and second time series data.

[0128] By connecting global hidden features, exogenous variables, and second time series data through the network residual link structure, predicted time series data corresponding to the first preset time period is obtained.

[0129] The residual connection structure can be used to accelerate network training. For each submodule M, the input x is M(x)+x after the residual connection.

[0130] It should be noted that this application also omits the Layer Norm layer in the naive Transformer architecture, and the time series data does not require position encoding.

[0131] The aforementioned preset model includes an encoder and a decoder. The encoder is used to compile the output of step 120 to speed up the data processing. The input to the decoder is the compiled exogenous variables and the first time series data, or the compiled exogenous variables and the original time series data.

[0132] To address the high time complexity of the dot product-based self-attention module in the Naive Transformer, this application modifies the dot product self-attention module in the Naive Transformer into a sparse self-attention module based on the locality-sensitive hash function.

[0133] To address the problem of inefficient autoregression during the inference phase in traditional time series models, exogenous variables obtained during preprocessing are used as input to the decoder. This allows the decoder to predict the required time series length in a multi-level, parallel manner during the inference phase, without the need for the masking mechanism in the naive attention decoder to prevent information leakage during the training phase. This effectively solves the problem of error accumulation in autoregressive prediction.

[0134] Specifically, such as Figure 3 As shown, the Transformer layer includes an N-level structure for the encoder and an M-level structure for the decoder.

[0135] The original time-series data is input into the encoder, passes through N accumulations of Locality Sensitive Hashpoint Attention, and is then input into the decoder. The decoder's input includes exogenous variables and the encoder's input. After M accumulations of Locality Sensitive Hashpoint Attention, the predicted time-series data corresponding to the first preset time period is obtained. Finally, the predicted time-series data is input into the Generative Adversarial Network (GAN). The Transformer layer acts as the generator in the GAN.

[0136] The encoder of this Transformer can specifically include:

[0137] The sparse self-attention submodule, the forward-linked network submodule, and the network residual link structure are used to implement step 130 above.

[0138] For its corresponding decoder, in addition to having the same sparse self-attention submodule, forward linking network submodule, and network residual linking structure as the encoder, there are the following modifications:

[0139] Unlike the naive Transformer, which directly right-shifts the input and uses it as the decoder's input, this application's decoder uses the exogenous variables obtained in the preprocessing stage as its input. This eliminates the need for a masking mechanism to prevent information leakage, as the periodic data is treated as an exogenous variable of the original time series, thus simplifying implementation. Furthermore, multi-level prediction can be parallelized during the inference stage, thereby abandoning the autoregressive structure of the naive Transformer during inference and significantly accelerating inference speed.

[0140] In addition to the sparse attention module for processing the upper-layer input, the decoder also includes an additional sparse self-attention module. Its corresponding V, K, and Q matrices are derived from the latent variables of the encoder output and the integration of the latent variables from the upper layer. This allows the decoder to obtain attention information from the latent variables of the input layer.

[0141] Attention information refers to the output of the encoder and the input of the decoder.

[0142] To achieve efficient and parallel implementation, a multi-turn local hash-sensitive self-attention mechanism is used. rounds To control the hyperparameters of the LSH cycle number, it means using n rounds Different hash functions Hash values ​​are computed in parallel to reduce the number of similar q values. i ,k j The possibility that they cannot be mapped to the same hash value. Specifically, for each h... r Its corresponding random matrix R is independent of other hash functions.

[0143] For naive self-attention computation, for each query q i ,make Represented as query q i The set of time locations involved.

[0144] Taking self-attention using masking as an example, it is... Obtain its associated attention vector:

[0145]

[0146] Among them, o i For attention vectors, P i For querying the matrix, k j Let z be the key matrix, and z be an intermediate variable in the operation. This formula will make the subsequent implementation based on parallel computing much easier.

[0147] Where i represents the position of the information corresponding to the i-th latent variable in the encoder, and j represents the position of the j-th latent variable in the decoder.

[0148] Here, the latent variables are calculated using the information compiled by the encoder, and the attention vector o is calculated based on the latent variables. i And for the attention vector o i Perform mathematical operations and input them into the decoder.

[0149] For multi-way locality-sensitive hashing, P i The base number is greatly reduced:

[0150] At the same time, in order to achieve parallelization, it is necessary to process each query q. i Sort i→s based on hash value and sequence position. i The sorted matrix is ​​then divided into m consecutive equal blocks, enabling parallel computation in multiple batches.

[0151] Here, for each hash function:

[0152] Each attention vector can then be written as

[0153] in,

[0154] as well as

[0155] Here, the attention vector is decomposed into m parts, thus the attention vector for each round... All of these can be computed independently in parallel. Current deep learning frameworks (such as PyTorch or Tensorflow) can easily construct the computational graph described above.

[0156] Here, on the one hand, locality-sensitive hashing is used; on the other hand, highly correlated operations are used to calculate the dot product of the query vector and the key vector, i.e., the above-mentioned calculation of the dot product of the query vector q. i Only for its neighboring key vector k j Calculating the dot product can further accelerate computational efficiency compared to the naive Transformer.

[0157] Step 140 is involved.

[0158] Based on the predicted time series data and the preset time series data, a preset model is trained until the preset model meets the preset training conditions, and a trained prediction model is obtained. The time series information of the predicted time series data matches the time series information of the preset time series data.

[0159] Among them, the preset time series data corresponds to the first time period, that is, the preset time series data is determined from the real sample data and is the time series data after the original time series data.

[0160] For example: the original time series data corresponds to the first week, the preset time series data corresponds to the second week, and the predicted time series data corresponds to the second week. Both the original time series data and the preset time series data are extracted from real sample data.

[0161] The preset training conditions can be that the preset model meets the preset convergence conditions, or that the loss value determined based on the predicted time series data and the preset time series data is less than the preset loss value.

[0162] This application employs adversarial training, enabling the model to capture sequence-level temporal relationships within time series data. Through adversarial training, the model used in this application exhibits greater generality and is less prone to overfitting compared to traditional models.

[0163] In one possible embodiment, step 140 may specifically include the following steps:

[0164] Calculate the adversarial loss value and mean squared loss value based on the predicted time series data and the preset time series data;

[0165] Based on the adversarial loss value and the mean squared loss value, a preset model is trained until the preset model meets the preset training conditions, thus obtaining the prediction model.

[0166] Among them, the preset time series data corresponds to the first preset time period.

[0167] The preset model includes a generator and a discriminator. The generator is used to generate the predicted time series data, and the discriminator is used to calculate the loss value based on the predicted time series data and the preset time series data.

[0168] Dot-product-based self-attention mechanisms can only capture point-to-point temporal relationships between multidimensional time series, failing to capture sequence-level temporal relationships, thus affecting the final performance of the model. This application employs an adversarial loss function based on generative adversarial networks (GAN Loss), combined with a mean squared error (MSE) loss function, to further reduce the learnable parameters of the model, enabling it to learn sequence-level temporal relationships.

[0169] The preset model includes a discriminator consisting of a two-layer feedforward neural network for classifying the true sequence Y. real The generated sequence Y obtained from the Transformer (and its corresponding generator) fake .

[0170] Let D be the discriminator, when the input is Y real Output 1 if the condition is met, otherwise output 0.

[0171] Specifically, the steps mentioned above for calculating the adversarial loss and mean squared loss based on predicted time-series data and preset time-series data may include the following steps:

[0172] Calculate the mean squared loss value based on the predicted time series data and the preset time series data;

[0173] The first predicted time series data is determined based on the predicted time series data and the original time series data;

[0174] The first preset timing data is determined based on the preset timing data and the original timing data;

[0175] The adversarial loss value is calculated based on the first predicted time series data and the first preset time series data.

[0176] Let the input of the Transformer be The output prediction of length τ is That is, predicting time series data.

[0177] but Where [a; b] represents combining vectors a and b into a single vector. The first predicted time series data is Y. fake .

[0178] make The adversarial loss function of the model is Ladv(θ) G θ D )=E[log(D(Y real ))]+E[log(D(Y fake The first preset time series data is Y. real .

[0179] Where, θ G For generator parameters, θ D Here, E represents the discriminator parameters, and E represents the mathematical expectation.

[0180] Let G be the Transformer generator described above, and the mean squared loss function of the model be...

[0181] Among them, Y i It is preset time-series data. It is predictive time series data.

[0182] Then L mse With L adv With mutually exclusive loss functions, the model optimization problem is transformed into a minmax problem: argmin G max D λL adv (θ G θ D )+L p (θ G ), where λ is a selection hyperparameter, ranging from [0, 1]. See [link to module architecture] for details. Figure 4 .

[0183] The training objective here is to make L mse With L adv They gradually decrease until the preset training conditions are met.

[0184] like Figure 4 As shown, the original time series data and exogenous variables are input into the generator. Through the operation of the encoder and decoder in the generator, the predicted time series data corresponding to the first preset time period is obtained. Based on the predicted time series data and the preset time series data, the mean square loss value is calculated.

[0185] Then, based on the predicted time series data and the original time series data, the first predicted time series data is determined; based on the preset time series data and the original time series data, the first preset time series data is determined; and based on the first predicted time series data and the first preset time series data, the adversarial loss value is calculated.

[0186] The discriminator consists of two layers of feedforward neural networks. During the model's training phase, the discriminator has a 50% probability of receiving a true multivariate time series Y. real There is a 0% probability of receiving Y generated by the generator (upstream Transformer model). fake .

[0187] The gradients of the generator and discriminator can be easily obtained using frameworks such as PyTorch and TensorFlow, and the model parameters can be learned using optimizers such as Adam. Specifically, the gradient of the generator G can be expressed as:

[0188]

[0189] in, For gradient calculation, Calculate the mathematical expectation.

[0190] The gradient of the discriminator D can be expressed as

[0191]

[0192] Using the automatic differential graph feature of deep learning frameworks, practitioners can easily calculate the gradients as described above. Optionally, practitioners can choose to apply L1 or L2 regularization to the model parameters. p Regularized training, the gradients of the generator and discriminator are respectively

[0193]

[0194]

[0195] Where α and β are the intensities of regularization, respectively.

[0196] In one possible embodiment, after step 140, the following steps may also be included:

[0197] Acquire third-order time series data;

[0198] The third time series data is preprocessed to obtain the exogenous variables corresponding to the third time series data. The exogenous variables are used to characterize the extension of the periodic data corresponding to the third time series data to a second preset time period.

[0199] The third time series data and the corresponding exogenous variables are input into the prediction model to obtain the fourth time series data corresponding to the second preset time period. The time series information of the fourth time series data matches the time series information of the exogenous variables corresponding to the third time series data.

[0200] After obtaining the prediction model, it can be used to predict data with prediction capabilities. Specifically, it can predict the third time series data to be processed, preprocess the third time series data to obtain the exogenous variables corresponding to the third time series data, and input the third time series data and exogenous variables into the prediction model to obtain the fourth time series data corresponding to the second preset time period.

[0201] The fourth time-series data predicted by the prediction model can be used in the following ways:

[0202] First, predicting operational monitoring data. Operational monitoring objectives generally include tasks such as hardware monitoring, system monitoring, application monitoring, network monitoring, traffic analysis, log monitoring, security monitoring, API monitoring, and performance monitoring. The most common examples are CPU and disk utilization. When disk and CPU usage is high, it may reduce application or system performance, or even cause incidents.

[0203] The algorithm can learn the growth trend of historical data for various indicators and predict the indicator situation in the future. Once it is found that the indicator value will exceed the warning line at a certain time point, an alarm will be issued. By predicting the operation and maintenance data, early warning can be achieved, reducing the system downtime rate and ensuring the efficient and stable operation of the IT system.

[0204] Second, there's the issue of capacity planning. Capacity planning is essentially the process of using strategies to predict system capacity, such as predicting when future load levels will saturate the system, and dynamically scaling the number of container pods, data volume, bandwidth, user scale, cloud storage capacity, and CPU capacity. In enterprises, each business is served by a series of different systems, and each business system is deployed on different machines.

[0205] The purpose of capacity planning is to enable each business system to clearly understand: when to add service nodes and when to remove them; and to determine the necessary scale of service expansion to ensure system availability and stability when facing business demands such as promotions, flash sales, and channel expansion. This helps avoid problems caused by insufficient system capacity when business volume increases.

[0206] Third, business data forecasting. This includes metrics such as commodity transaction volume and system traffic. Business forecasting is a tool used by businesses to predict future business development (revenue, sales, resource requirements, and product data) based on time-series data. Accurate forecasts provide data support for strategic decision-making, enabling effective budgeting and resource allocation, and helping senior management make more rational plans.

[0207] In this embodiment, the original time-series data is preprocessed to obtain exogenous variables that characterize the periodic data corresponding to the original time-series data extended to a first preset time period. Here, the periodic data corresponding to the original time-series data can be incorporated into the exogenous variables, facilitating the integration of domain-specific features from the actual usage scenario into the subsequent prediction process. Then, the original time-series data and exogenous variables are input into a preset model. Based on the exogenous variables and a locality-sensitive hash function, the original time-series data is processed to obtain predicted time-series data corresponding to the first preset time period. Since the exogenous variables are information extracted from the original time-series data, autoregressive inference is unnecessary during model inference, enabling multi-level parallel inference and accelerating the inference speed. Finally, the preset model is trained based on the predicted time-series data and the preset time-series data until the preset model meets preset training conditions, resulting in a trained prediction model. Therefore, the trained prediction model can quickly and accurately predict the predicted time-series data corresponding to the original time-series data.

[0208] Based on the above Figure 1 The present application also provides a training apparatus for the prediction model, as shown in the training method of the prediction model. Figure 5 As shown, the training device 500 for the prediction model may include:

[0209] The acquisition module 510 is used to acquire raw time series data.

[0210] The preprocessing module 520 is used to preprocess the original time series data to obtain exogenous variables. The exogenous variables are used to characterize the extension of the periodic data corresponding to the original time series data to a first preset time period.

[0211] The input module 530 is used to input the original time series data and exogenous variables into the preset model, and process the original time series data based on the exogenous variables and the locality-sensitive hash function to obtain the predicted time series data corresponding to the first preset time period.

[0212] The training module 540 is used to train a preset model based on the predicted time series data and the preset time series data until the preset model meets the preset training conditions, and obtain a trained prediction model. The time series information of the predicted time series data matches the time series information of the preset time series data.

[0213] In one possible embodiment, the preprocessing module 520 is specifically used to: determine the association information of the original time series data, the association information including: change point annotation information and holiday information in the original time series data;

[0214] Based on the original time series data and related information, the first time series data is determined;

[0215] The first time series data is input into the time series decomposer to obtain exogenous variables.

[0216] In one possible embodiment, the preprocessing module 520 is specifically used to: input the first time series data into the time series decomposer and extract the parameters corresponding to the Nth order Fourier series of the correlation information, where N is a positive integer;

[0217] Numerical simulations were performed on the parameters to obtain exogenous variables.

[0218] In one possible embodiment, the input module 530 is specifically used to: input the original time series data and exogenous variables into a preset model, and process the original time series data based on a locality-sensitive hash function to obtain second time series data;

[0219] Global latent features are determined based on exogenous variables and second time-series data;

[0220] Based on global hidden features, exogenous variables, and second time series data, predictive time series data corresponding to the first preset time period are determined.

[0221] In one possible embodiment, the device 500 may further include:

[0222] The determination module is used to: determine the query matrix and key matrix based on exogenous variables;

[0223] Determine the locality-sensitive hash function based on the query matrix and the key matrix.

[0224] In one possible embodiment, the input module 530 is specifically used for:

[0225] Determine the random matrix based on the data dimensions of the original time series data;

[0226] Based on the random matrix, determine the weight parameter matrix and the bias parameter vector;

[0227] Based on the weight parameter matrix and the bias parameter vector, global latent features are obtained by calculating exogenous variables and second time series data.

[0228] In one possible embodiment, the training module 540 is specifically used for:

[0229] Based on the predicted time series data and the preset time series data, train the preset model until the preset model meets the preset training conditions, and obtain the trained prediction model, including:

[0230] Calculate the adversarial loss value and mean squared loss value based on the predicted time series data and the preset time series data;

[0231] Based on the adversarial loss value and the mean squared loss value, a preset model is trained until the preset model meets the preset training conditions, thus obtaining the prediction model.

[0232] In one possible embodiment, the training module 540 is specifically used for:

[0233] Calculate the mean squared loss value based on the predicted time series data and the preset time series data;

[0234] The first predicted time series data is determined based on the predicted time series data and the original time series data;

[0235] The first preset timing data is determined based on the preset timing data and the original timing data;

[0236] The adversarial loss value is calculated based on the first predicted time series data and the first preset time series data.

[0237] In one possible embodiment, the device 500 may further include: a prediction module, specifically configured to:

[0238] Acquire third-order time series data;

[0239] The third time series data is preprocessed to obtain the exogenous variables corresponding to the third time series data. The exogenous variables are used to characterize the extension of the periodic data corresponding to the third time series data to a second preset time period.

[0240] The third time series data and the corresponding exogenous variables are input into the prediction model to obtain the fourth time series data corresponding to the second preset time period. The time series information of the fourth time series data matches the time series information of the exogenous variables corresponding to the third time series data.

[0241] In this embodiment, the original time-series data is preprocessed to obtain exogenous variables that characterize the periodic data corresponding to the original time-series data extended to a first preset time period. Here, the periodic data corresponding to the original time-series data can be incorporated into the exogenous variables, facilitating the integration of domain-specific features from the actual usage scenario into the subsequent prediction process. Then, the original time-series data and exogenous variables are input into a preset model. Based on the exogenous variables and a locality-sensitive hash function, the original time-series data is processed to obtain predicted time-series data corresponding to the first preset time period. Since the exogenous variables are information extracted from the original time-series data, autoregressive inference is unnecessary during model inference, enabling multi-level parallel inference and accelerating the inference speed. Finally, the preset model is trained based on the predicted time-series data and the preset time-series data until the preset model meets preset training conditions, resulting in a trained prediction model. Therefore, the trained prediction model can quickly and accurately predict the predicted time-series data corresponding to the original time-series data.

[0242] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application is shown.

[0243] An electronic device may include a processor 601 and a memory 602 storing computer program instructions.

[0244] Specifically, the processor 601 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0245] Memory 602 may include a large-capacity memory for data or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), a floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 602 may include removable or non-removable (or fixed) media. Where appropriate, memory 602 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 602 is a non-volatile solid-state memory. In a particular embodiment, memory 602 includes read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0246] The processor 601 implements any of the prediction model training methods in the embodiment shown in the figure by reading and executing computer program instructions stored in the memory 602.

[0247] In one example, the electronic device may also include a communication interface 606 and a bus 610. Wherein, as... Figure 6 As shown, the processor 601, memory 602, and communication interface 606 are connected through bus 610 and complete communication with each other.

[0248] The communication interface 606 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0249] Bus 610 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 610 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0250] The electronic device can execute the training method of the prediction model in the embodiments of this application, thereby achieving the combination of Figures 1-4 The training method for the prediction model is described.

[0251] Furthermore, in conjunction with the training method of the prediction model in the above embodiments, this application embodiment can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; these computer program instructions are implemented when executed by a processor. Figures 1-4 The training method for the prediction model is described.

[0252] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0253] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0254] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0255] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for training a prediction model, characterized in that, The method includes: Obtain raw time-series data; wherein, the raw time-series data is data related to time series, including: network throughput corresponding to time series, disk utilization corresponding to time series, or CPU utilization corresponding to time series; The original time series data is preprocessed to obtain exogenous variables, which are used to characterize the extension of the periodic data corresponding to the original time series data to a first preset time period. The original time series data and the exogenous variables are input into a preset model. Based on the exogenous variables and the locality-sensitive hash function, the original time series data is processed to obtain the predicted time series data corresponding to the first preset time period. Based on the predicted time series data and the preset time series data, the preset model is trained until the preset model meets the preset training conditions, and a trained prediction model is obtained. Acquire third-order time series data; The third time series data is preprocessed to obtain the exogenous variables corresponding to the third time series data. The exogenous variables are used to characterize the extension of the periodic data corresponding to the third time series data to a second preset time period. The third time series data and the exogenous variables corresponding to the third time series data are input into the prediction model to obtain the fourth time series data corresponding to the second preset time period. The preset model includes a sparse self-attention submodule, a forward-linked network submodule, and a network residual link structure. The original time-series data and the exogenous variables are input into the preset model. Based on the exogenous variables and a locality-sensitive hash function, the original time-series data is processed to obtain predicted time-series data corresponding to the first preset time period, including... The original time series data is input into the sparse self-attention submodule, and the original time series data is processed based on the locality-sensitive hash function to obtain the second time series data. The exogenous variables and the second time-series data are input into the forward linking network submodule to capture global latent features; The predicted time series data is obtained by connecting the global latent features, the exogenous variables, and the second time series data through the network residual link structure.

2. The method according to claim 1, characterized in that, The preprocessing of the original time-series data to obtain exogenous variables includes: Determine the association information of the original time series data, the association information including: change point annotation information and holiday information in the original time series data; The first time series data is determined based on the original time series data and the associated information; The first time series data is input into the time series decomposer to obtain the exogenous variable.

3. The method according to claim 2, characterized in that, The step of inputting the first time-series data into the time-series decomposer to obtain the exogenous variables includes: The first time series data is input into the time series decomposer to extract the parameters corresponding to the Nth order Fourier series of the correlation information, where N is a positive integer; The exogenous variables are obtained by performing numerical simulations on the parameters.

4. The method according to claim 1, characterized in that, Before inputting the original time series data into the sparse self-attention submodule and processing the original time series data based on the locality-sensitive hash function to obtain the second time series data, the method further includes: Based on the exogenous variables, determine the query matrix and the key matrix; The locality-sensitive hash function is determined based on the query matrix and the key matrix.

5. The method according to claim 1, characterized in that, The step of inputting the exogenous variable and the second time-series data into the feedforward network submodule to capture the global latent features includes: Determine the random matrix based on the data dimensions of the original time-series data; Based on the random matrix, determine the weight parameter matrix and the bias parameter vector; The global latent features are obtained by calculating the exogenous variables and the second time series data based on the weight parameter matrix and the bias parameter vector.

6. The method according to claim 1, characterized in that, The step of training a preset model based on the predicted time-series data and preset time-series data until the preset model meets preset training conditions to obtain a trained prediction model includes: Based on the predicted time series data and the preset time series data, calculate the adversarial loss value and the mean squared loss value; The preset model is trained based on the adversarial loss value and the mean squared loss value until the preset model meets the preset training conditions, thereby obtaining the prediction model.

7. The method according to claim 6, characterized in that, The step of calculating the adversarial loss value and the mean squared loss value based on the predicted time series data and the preset time series data includes: The mean squared loss value is calculated based on the predicted time series data and the preset time series data. Based on the predicted time series data and the original time series data, the first predicted time series data is determined; The first preset timing data is determined based on the preset timing data and the original timing data; The adversarial loss value is calculated based on the first predicted time series data and the first preset time series data.

8. A method and apparatus for training a prediction model, characterized in that, The training device for the prediction model includes: The acquisition module is used to acquire raw time-series data; wherein, the raw time-series data is data related to the time series, including: network throughput corresponding to the time series, disk utilization corresponding to the time series, or CPU utilization corresponding to the time series; The preprocessing module is used to preprocess the original time series data to obtain exogenous variables, which are used to characterize the extension of the periodic data corresponding to the original time series data to a first preset time period. The input module is used to input the original time series data and the exogenous variables into a preset model, and process the original time series data based on the exogenous variables and the locality-sensitive hash function to obtain the predicted time series data corresponding to the first preset time period. The training module is used to train the preset model based on the predicted time series data and the preset time series data until the preset model meets the preset training conditions, thereby obtaining a trained prediction model. The time series information of the predicted time series data is matched with the time series information of the preset time series data. The prediction module is used for: Acquire third-order time series data; The third time series data is preprocessed to obtain the exogenous variables corresponding to the third time series data. The exogenous variables are used to characterize the extension of the periodic data corresponding to the third time series data to a second preset time period. The third time series data and the corresponding exogenous variables are input into the prediction model to obtain the fourth time series data corresponding to the second preset time period. The time series information of the fourth time series data matches the time series information of the exogenous variables corresponding to the third time series data. The preset model includes: a sparse self-attention submodule, a forward-linked network submodule, and a network residual connection structure. The input module is specifically used for: The original time series data is input into the sparse self-attention submodule, and the original time series data is processed based on the locality-sensitive hash function to obtain the second time series data. The exogenous variables and the second time-series data are input into the forward linking network submodule to capture global latent features; The predicted time series data is obtained by connecting the global latent features, the exogenous variables, and the second time series data through the network residual link structure.

9. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the method as described in any one of claims 1-7.

Images