Multivariable long-time sequence prediction method and device based on deep learning

Abstract

The invention discloses a deep learning-based multivariable long-time sequence prediction method and device, proposes a TLform model, adopts a simpler bilinear model architecture, and carries out time and latitude normalization on the whole data when the model is input, and carries out reverse normalization when the model is output. A CNN-Attention mechanism is adopted, convolution is carried out on the time dimension, the time and variable latitude of input data are exchanged, a deep learning network learns the relation between variables and then predicts the time and latitude, and therefore the multivariable prediction capacity of a model is greatly improved through modification. Meanwhile, aiming at the problem that an Attention mechanism is insensitive to learning of trend terms, the model adopts a sliding window to divide original data into trend terms, seasonal terms and residual terms, training is carried out respectively, and finally training results are added to obtain a final result. A large number of experiments show that the prediction precision of the model is superior to that of most current time sequence prediction models.

Description

Technical Field

[0001] The present invention relates to the technical field of time prediction based on deep learning, and in particular to a multivariate long time series prediction method and device based on deep learning. Background Art

[0002] Time series prediction has a wide range of applications in many fields. Time series prediction is extremely challenging, especially long-term time series prediction. Traditional Recurrent models such as RNN and LSTM are susceptible to gradient vanishing and explosion during training, especially when facing long time series. Moreover, such Recurrent models cannot be calculated in parallel, which limits their application to large-scale problems. The Transformer model has achieved great success in natural language processing and computer vision, so some scholars have applied it to time series prediction, taking advantage of its attention mechanism's ability to capture long-term dependencies, and has dominated the research on long-term prediction based on neural networks in the past few years.

[0003] However, the original Transformer model still has various problems in the prediction process. For example, when predicting long time series, the memory occupied will increase exponentially with the increase of prediction time. The prediction effect for non-stationary time series data is not good, the prediction ability for trend items is poor, multiple variables will affect each other during the prediction process, and the influence of time encoding and position encoding on variables, etc. Therefore, various variant models based on the Transformer structure have been proposed one after another. The first is some models that improve the problem of excessive memory usage for training, such as Reformer, informer, Autoformer, Fedformer models, etc. These models improve the computational efficiency by modifying the Attention mechanism. The second is a model proposed for the original data to solve the problem of multiple variables affecting each other in the multivariate prediction process of the Transformer model, such as PatchTST, NSTransformer, etc. The third is a model that improves the structure of the Transformer model, such as the Crossformer model. However, the models in the prior art have not yet been able to take into account both efficiency and accuracy. Summary of the invention

[0004] The present invention provides a multivariate long time series prediction method and device based on deep learning, which are used to solve or at least partially solve the technical problem in the prior art that it is impossible to take into account both computing efficiency and prediction accuracy at the same time.

[0005] In order to solve the above technical problems, the first aspect of the present invention provides a multivariate long time series prediction method based on deep learning, comprising:

[0006] Get the original time series and build a training set;

[0007] Based on the Tansformer model, a multivariate long time series prediction model TLformer is constructed. The multivariate long time series prediction model adopts a structure combining an encoder and a linear layer.

[0008] The constructed training set is used to train the multivariate long time series prediction model. During the training process, the model uses a sliding window method to divide the original time series in the training set into a trend term and a seasonal term plus a residual term. The trend term enters the linear layer structure for training, and the seasonal term plus the residual term enters the encoder structure for learning. Finally, the output results of the two structures are added to obtain the final output.

[0009] Use the trained model to perform multivariate long time series forecasting.

[0010] In one embodiment, the multivariate long time series prediction model includes a normalization layer for normalizing the input time series in the time dimension.

[0011] In one implementation, the linear layer structure includes M linear layers, a normalization layer, and a mapping layer. After the trend item enters the linear layer structure for training, a first output is obtained.

[0012] In one embodiment, the encoder structure includes a CNN-Attention layer, a feedforward neural network layer and a mapping layer, wherein the CNN-Attention layer is a convolution-based attention layer, including a dimensionality transformation function, a convolution layer and a self-attention mechanism layer.

[0013] In one embodiment, before the seasonal term plus the residual term enters the encoder structure for learning, the method further includes:

[0014] Perform correlation analysis on the variables of the original time series. If the correlation between variables is strong, add position coding and time coding to the seasonal term plus residual term of the original time series. Otherwise, use a linear layer to process the seasonal term plus residual term of the original time series.

[0015] In one implementation, the seasonal term plus the residual term enters the encoder structure for learning, including:

[0016] The variable dimension and time dimension of the seasonal term plus the residual term of the input time series are exchanged.

[0017] Perform convolution operation on the time dimension to obtain the convolution result X cnn ;

[0018] X cnnCopy it into three copies, denoted as Q, K, and V, where Q is the query vector, K is the key vector, and V is the numerical vector. Input Q, K, and V into the self-attention mechanism layer to obtain the attention result X attn , expressed as d is the dimension of the input data, Softmax is the activation function, X attn =Attn(Q,K,V);

[0019] Through the residual network, X attn With X cnn Add and normalize to get the first intermediate result X nor ;

[0020] X nor The second intermediate result X is obtained by processing the feedforward neural network layer. feed ;

[0021] Through the residual network, X feed Then with X nor After addition, normalization is performed and then mapped to the required output dimension through the mapping layer to obtain the second output X out2 .

[0022] In one implementation, the output results of the two structures are finally added together to obtain a final output, including:

[0023] The first output obtained by the linear structure is added to the second output obtained by the encoder structure, and then processed by the denormalization layer to obtain the final output.

[0024] Based on the same inventive concept, the second aspect of the present invention provides a multivariate long time series prediction device based on deep learning, comprising:

[0025] Data acquisition module, used to obtain the original time series and build the training set;

[0026] The model building module is used to build a multivariate long time series prediction model TLformer based on the Tansformer model. The multivariate long time series prediction model adopts a structure combining an encoder and a linear layer;

[0027] The training module is used to train the multivariate long time series prediction model using the constructed training set. During the training process, the model uses a sliding window method to divide the original time series in the training set into a trend term and a seasonal term plus a residual term. The trend term enters the linear layer structure for training, and the seasonal term plus the residual term enters the encoder structure for learning. Finally, the output results of the two structures are added to obtain the final output;

[0028] The forecasting module is used to perform multivariate long time series forecasting using the trained model.

[0029] Based on the same inventive concept, the third aspect of the present invention provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the method described in the first aspect is implemented.

[0030] Based on the same inventive concept, the fourth aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0031] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows:

[0032] The present invention proposes a multivariate long time series prediction method based on deep learning and constructs a multivariate long time series prediction model Tlformer. The model modifies the traditional Tansformer model structure, abandons the Encoder+Decoder structure, and adopts the Encoder+linear layer (encoder combined with linear layer) structure, which greatly reduces the computing memory and computing time in the training process.

[0033] Furthermore, in order to address the problem that the Attention mechanism is insensitive to the trend item learning in the time series, a sliding window method is used in the TLformer model to divide the original time series data into a trend item and a seasonal item plus a residual item. The trend item enters the linear layer structure for training, and the seasonal item plus the residual item enters the Encoder structure for learning. The training results of the two structures are added together to obtain the final output.

[0034] Furthermore, in order to solve the problem of poor accuracy of predicting non-stationary time series data by traditional Transformer models, this model uses normalization operation in the time dimension, so that the input of the entire data obeys the same mean and variance, and the distribution difference of the overall training samples in the time dimension is weakened, thereby improving the stability of the input data. Although this enhances the predictability of the data, it also damages the original data distribution. Therefore, when outputting, the calculated mean and variance are used to re-scale the results in reverse order to restore the information lost during normalization.

[0035] Furthermore, in order to solve the problem that adding time coding and position coding to variables affects the relationship between variables, this model adopts different processing strategies for different situations. For the case where the relationship between variables is strong, by adding time coding and position coding, the time relationship before and after the single variable is strengthened, and the added time coding and position coding will not have much impact on finding the relationship between variables. For the case where the relationship between variables is weak or there is no relationship, the present invention uses a linear layer to replace the time coding and position coding to prevent the added coding data from forcibly adding the relationship between the variables and affecting the learning effect of the variables.

[0036] Furthermore, the present invention modifies the learning target of the Attention mechanism, transforms the dimensions of the Q, K, and V variables originally passed into the Attention mechanism, exchanges the time feature dimension with the variable feature dimension, and changes the original calculation of the score of the variable feature on time to the calculation of the score of the time feature on the variable, that is, first learn the related features between variables, and then learn the time features. This can effectively reduce the impact of the relationship between variables in the time learning process.

[0037] Furthermore, the present invention modifies the point-to-point prediction mechanism of the Attention mechanism, and convolves the incoming Q, K, and V variables in the time dimension, first convolving the time features of adjacent time points and then entering the Attention mechanism for learning. This transforms the point-to-point prediction into a line-to-line prediction, making the prediction result more scientific and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 It is a structural diagram of the multivariate long time series prediction model Tlformer in an embodiment of the present invention;

[0040] Figure 2 4 is a structural diagram of a feed-forward neural network layer in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] Due to the great potential of the traditional Transformer model in visual tasks such as image classification and detection, some researchers have applied this architecture to time series prediction. However, there are some problems with this model architecture in the task of time series prediction. The present invention is a deep learning model for time series prediction-TLformer built using the pytorch deep learning framework and pycharm development tools. The model abandons the architecture of the traditional Transformer model and changes to a simpler bilinear model architecture, which greatly reduces the memory and time of model training. In order to improve the prediction ability of non-stationary data, the model normalizes the entire data in the time dimension when inputting, and denormalizes it when outputting. At the same time, the Attention mechanism is improved. The traditional Attention mechanism is a point-to-point prediction, while the CNN-Attention mechanism adopted by this model performs convolution in the time dimension, turning the point-to-point prediction into a line-to-line prediction, making the prediction result more accurate, and exchanging the time and variable dimensions of the input data, so that the deep learning network first learns the relationship between the variables, and then predicts the time dimension. This modification greatly increases the ability of the model to predict multiple variables. At the same time, in order to address the problem that the Attention mechanism is not sensitive to the learning of trend items, the model uses a sliding window to divide the original data into trend items and seasonal items plus residual items, trains them separately, and finally adds the training results to obtain the final result. A large number of experiments have shown that the prediction accuracy of this model is better than that of most current time series prediction models.

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] Embodiment 1

[0044] The embodiment of the present invention provides a multivariate long time series prediction method based on deep learning, comprising:

[0045] Get the original time series and build a training set;

[0046] Based on the Tansformer model, a multivariate long time series prediction model TLformer is constructed. The multivariate long time series prediction model adopts a structure combining an encoder and a linear layer.

[0047] The constructed training set is used to train the multivariate long time series prediction model. During the training process, the model uses a sliding window method to divide the original time series in the training set into a trend term and a seasonal term plus a residual term. The trend term enters the linear layer structure for training, and the seasonal term plus the residual term enters the encoder structure for learning. Finally, the output results of the two structures are added to obtain the final output.

[0048] Use the trained model to perform multivariate long time series forecasting.

[0049] Different from the traditional Encoder+Decoder structure, the model of the present invention adopts a structure combining an encoder and a linear layer: Encoder+linear layer structure, which greatly reduces the computing memory and computing time in the training process.

[0050] In one embodiment, the multivariate long time series prediction model includes a normalization layer for normalizing the input time series in the time dimension.

[0051] By normalizing the input time series in the time dimension, the entire data input is made to obey the same mean and variance, and the distribution differences of the overall training samples in the time dimension are weakened, thereby improving the stability of the input data.

[0052] In one implementation, the linear layer structure includes M linear layers, a normalization layer, and a mapping layer. After the trend item enters the linear layer structure for training, a first output is obtained.

[0053] The number of M can be set according to actual conditions.

[0054] In one embodiment, the encoder structure includes a CNN-Attention layer, a feedforward neural network layer and a mapping layer, wherein the CNN-Attention layer is a convolution-based attention layer including a dimensionality transformation function, a convolution layer and a self-attention mechanism layer.

[0055] Specifically, the dimension transformation function is mainly used to exchange the variable dimension and the time dimension, the convolution layer is used to extract the time dimension information from the time dimension, and the self-attention mechanism layer uses the attention mechanism to process the input feature vector.

[0056] In one embodiment, before the seasonal term plus the residual term enters the encoder structure for learning, the method further includes:

[0057] Perform correlation analysis on the variables of the original time series. If the correlation between variables is strong, add position coding and time coding to the seasonal term plus residual term of the original time series. Otherwise, use a linear layer to process the seasonal term plus residual term of the original time series.

[0058] Specifically, in order to solve the problem that adding time coding and position coding to variables affects the relationship between variables, the present invention adopts different processing strategies for different situations. The first situation: the relationship between variables is strong. At this time, by adding time coding and position coding, the time relationship before and after the single variable is strengthened, and the added coding will not have much impact on finding the relationship between variables. The second situation: the relationship between variables is weak or there is no relationship between them. By using linear layers to replace time coding and position coding, it is prevented that the added coding data will forcibly increase the correlation of the variables and affect the learning effect of the variables.

[0059] In one implementation, the seasonal term plus the residual term enters the encoder structure for learning, including:

[0060] The variable dimension and time dimension of the seasonal term plus the residual term of the input time series are exchanged.

[0061] Perform convolution operation on the time dimension to obtain the convolution result X cnn ;

[0062] X cnn Copy it into three copies, denoted as Q, K, and V, where Q is the query vector, K is the key vector, and V is the numerical vector. Input Q, K, and V into the self-attention mechanism layer to obtain the attention result X attn , expressed as d is the dimension of the input data, Softmax is the activation function, X attn =Attn(Q,K,V);

[0063] Through the residual network, X attn With X cnn Add and normalize to get the first intermediate result X nor ;

[0064] X nor The second intermediate result X is obtained by processing the feedforward neural network layer. feed ;

[0065] Through the residual network, X feed Then with X nor After addition, normalization is performed and then mapped to the required output dimension through the mapping layer to obtain the second output X out2 .

[0066] The mapping layer is a projection layer, that is, a regular, dense, linear layer without an activation function, which will eventually expand the dimensions to the required dimensions through the results of the above structures.

[0067] Specifically, the CNN-Attention layer of the present invention modifies the learning goal of the Attention mechanism, transforms the dimensions of the Q, K, and V variables originally passed into the Attention mechanism, exchanges the time feature dimension with the variable feature dimension, and changes the original calculation of the score of the variable feature in time to the calculation of the score of the time feature in the variable, that is, first learn the related features between variables, and then learn the time features. This can effectively reduce the impact of the relationship between variables in the time learning process. In addition, the point-to-point prediction mechanism of the Attention mechanism is also modified, and the Q, K, and V variables passed in are convolved in the time dimension. The time features of adjacent time points are first convolved and then enter the Attention mechanism for learning. This changes the point-to-point prediction into a line-to-line prediction, making the prediction result more scientific and accurate.

[0068] In one implementation, the output results of the two structures are finally added together to obtain a final output, including:

[0069] The first output obtained by the linear structure is added to the second output obtained by the encoder structure, and then processed by the denormalization layer to obtain the final output.

[0070] Specifically, although the predictability of data is enhanced by using normalization operation in the time dimension, the original data distribution is also damaged. Therefore, the present invention uses the calculated mean and variance to perform reverse scaling (processing of the anti-normalization layer) on the result when outputting it to restore the information lost during normalization.

[0071] The present invention belongs to the field of time prediction based on deep learning. It is a deep learning model for time series prediction based on the pytorch deep learning framework and the pycharm development tool, referred to as TLformer. The structure diagram of the model is shown in the attached figure. Figure 1 .

[0072] The TLformer model has the ability to predict multiple variables and long time series. Compared with some current deep learning models, the prediction accuracy of this model has been greatly improved. Take the WTH data as an example for demonstration. The WTH dataset contains local climate data from nearly 1,600 locations in the United States. It is a 1-hour dataset spanning 4 years from 2010 to 2013. Each data point consists of a target value "WetBulbCelsius" and 11 climate features. 60% of the data is used as a training set, 10% as a validation dataset, and 20% as a test dataset. Some training parameters are as follows:

[0073] Table 1 Parameter settings for training part of TLformer model

[0074]

[0075] The general process of training the model using the training set is as follows:

[0076] 1. Normalize the time dimension of the input data (time series) X.

[0077] 2. Use a sliding window to separate the input data into a trend term and a seasonal term plus a residual term.

[0078] 3. The trend item enters M linear layers, normalization layers, and mapping layers for training, and finally obtains X out1 .

[0079] 4. Seasonal term plus residual term Through the correlation analysis of variables, the corresponding encoding method is adopted: if the correlation is strong, the position encoding layer and time encoding layer are turned on, and if the correlation is weak, the linear layer is used. Then enter the CNN-Attention layer (the processing process includes 4.1~4.2).

[0080] 4.1: First, the input data (seasonal term plus residual term) is swapped in dimensions, the variable dimension and the time dimension are swapped, and then the time dimension is convolved to extract the time dimension information to obtain X cnn , copy the data into three copies, denoted as Q, K, and V respectively.

[0081] 4.2: Pass Q, K, and V through the self-attention mechanism layer to obtain X attn , d is the dimension of the input data and Softmax is the activation function.

[0082] 5. Through the residual network, X attn With X cnn Add and normalize to get X nor .

[0083] 6. X nor Then, we get X through the feed-forward neural network layer. feed The layer structure is shown in the attached figure Figure 2 .

[0084] 7. Through the residual network, make X feed Then with X nor Add and normalize, and map to the required output dimension through the Projection layer to get X out2 .

[0085] 8. X out1 With X out2 Add and pass through the anti-normalization layer to offset the normalization of the time dimension in step 2. The final result is X out .

[0086] The validation set is used to determine whether the training is complete. If the loss of the validation set increases for three consecutive epochs, it indicates that the training is complete. Finally, the model training accuracy is verified on the test set. The accuracy of some current time series prediction models is compared with that of TLformer, and the results are shown in Table 2.

[0087] Table 2 Comparison of model multivariate prediction accuracy

[0088]

[0089] Embodiment 2

[0090] Based on the same inventive concept, this embodiment discloses a multivariate long time series prediction device based on deep learning, comprising:

[0091] Data acquisition module, used to obtain the original time series and build the training set;

[0092] The model building module is used to build a multivariate long time series prediction model TLformer based on the Tansformer model. The multivariate long time series prediction model adopts a structure combining an encoder and a linear layer;

[0093] The training module is used to train the multivariate long time series prediction model using the constructed training set. During the training process, the model uses a sliding window method to divide the original time series in the training set into a trend term and a seasonal term plus a residual term. The trend term enters the linear layer structure for training, and the seasonal term plus the residual term enters the encoder structure for learning. Finally, the output results of the two structures are added to obtain the final output;

[0094] The forecasting module is used to perform multivariate long time series forecasting using the trained model.

[0095] Since the device introduced in the second embodiment of the present invention is a device used to implement the multivariate long time series prediction method based on deep learning in the first embodiment of the present invention, based on the method introduced in the first embodiment of the present invention, the person skilled in the art can understand the specific structure and deformation of the device, so it is not repeated here. All devices used in the method in the first embodiment of the present invention belong to the scope of protection of the present invention.

[0096] Embodiment 3

[0097] Based on the same inventive concept, the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed, the method described in the first embodiment is implemented.

[0098] Since the computer-readable storage medium introduced in the third embodiment of the present invention is a computer-readable storage medium used to implement the multivariate long time series prediction method based on deep learning in the first embodiment of the present invention, based on the method introduced in the first embodiment of the present invention, those skilled in the art can understand the specific structure and deformation of the computer-readable storage medium, so it is not repeated here. All computer-readable storage media used in the method of the first embodiment of the present invention belong to the scope of protection of the present invention.

[0099] Embodiment 4

[0100] Based on the same inventive concept, the present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the method in Embodiment 1 is implemented when the processor executes the above program.

[0101] Since the computer device introduced in the fourth embodiment of the present invention is a computer device used to implement the multivariate long time series prediction method based on deep learning in the first embodiment of the present invention, based on the method introduced in the first embodiment of the present invention, the person skilled in the art can understand the specific structure and deformation of the computer device, so it is not repeated here. All computer devices used in the method in the first embodiment of the present invention belong to the scope of protection of the present invention.

[0102] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0103] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0104] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic creative concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if these modifications and variations of the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A multivariate long time series prediction method based on deep learning, characterized in that: include: Get the original time series and build a training set; Based on the Tansformer model, a multivariate long time series prediction model TLformer is constructed. The multivariate long time series prediction model adopts a structure combining an encoder and a linear layer. The constructed training set is used to train the multivariate long time series prediction model. During the training process, the model uses a sliding window method to divide the original time series in the training set into a trend term and a seasonal term plus a residual term. The trend term enters the linear layer structure for training, and the seasonal term plus the residual term enters the encoder structure for learning. Finally, the output results of the two structures are added to obtain the final output. Use the trained model to perform multivariate long time series forecasting.

2. The multivariate long time series prediction method based on deep learning as claimed in claim 1, characterized in that: The multivariate long time series prediction model includes a normalization layer for normalizing the input time series in the time dimension.

3. The multivariate long time series prediction method based on deep learning as claimed in claim 1, characterized in that: The linear layer structure includes M linear layers, a normalization layer and a mapping layer. After the trend item enters the linear layer structure for training, the first output is obtained.

4. The multivariate long time series prediction method based on deep learning as claimed in claim 1, characterized in that: The encoder structure includes a CNN-Attention layer, a feedforward neural network layer, and a mapping layer, wherein the CNN-Attention layer is a convolution-based attention layer, including a dimension transformation function, a convolution layer, and a self-attention mechanism layer.

5. The multivariate long time series prediction method based on deep learning as claimed in claim 1, characterized in that: Before the seasonal term plus the residual term enters the encoder structure for learning, the method further includes: Perform correlation analysis on the variables of the original time series. If the correlation between variables is strong, add position coding and time coding to the seasonal term plus residual term of the original time series. Otherwise, use a linear layer to process the seasonal term plus residual term of the original time series.

6. The multivariate long time series prediction method based on deep learning as claimed in claim 4, characterized in that: The seasonal term plus the residual term enters the encoder structure for learning, including: The variable dimension and time dimension of the seasonal term plus the residual term of the input time series are exchanged. Perform convolution operation on the time dimension to obtain the convolution result X cnn ; X cnn Copy it into three copies, denoted as Q, K, and V, where Q is the query vector, K is the key vector, and V is the numerical vector. Input Q, K, and V into the self-attention mechanism layer to obtain the attention result X attn , expressed as d is the dimension of the input data, Softmax is the activation function, X attn =Attn(Q,K,V); Through the residual network, X attn With X cnn Add and normalize to get the first intermediate result X nor ; X nor The second intermediate result X is obtained by processing the feedforward neural network layer. feed ; Through the residual network, X feed Then with X nor After addition, normalization is performed and then mapped to the required output dimension through the mapping layer to obtain the second output X out2 .

7. The multivariate long time series prediction method based on deep learning as claimed in claim 1, characterized in that: Finally, the output results of the two structures are added together to obtain the final output, including: The first output obtained by the linear structure is added to the second output obtained by the encoder structure, and then processed by the denormalization layer to obtain the final output.

8. A multivariate long time series prediction device based on deep learning, characterized in that: include: Data acquisition module, used to obtain the original time series and build the training set; The model building module is used to build a multivariate long time series prediction model TLformer based on the Tansformer model. The multivariate long time series prediction model adopts a structure combining an encoder and a linear layer; The training module is used to train the multivariate long time series prediction model using the constructed training set. During the training process, the model uses a sliding window method to divide the original time series in the training set into a trend term and a seasonal term plus a residual term. The trend term enters the linear layer structure for training, and the seasonal term plus the residual term enters the encoder structure for learning. Finally, the output results of the two structures are added to obtain the final output; The forecasting module is used to perform multivariate long time series forecasting using the trained model.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

Images