Time sequence prediction method based on deep learning model

Abstract

The invention discloses a time sequence prediction method based on a deep learning model, and the method mainly comprises the steps: firstly carrying out the preprocessing of sequence data, recognizing and processing the missing value and abnormal value in the time sequence data, guaranteeing the data quality, and carrying out the extraction of a timestamp; then decomposing the original time sequence into a plurality of subsequences with different frequencies by using a variational mode decomposition (VMD) model, effectively distinguishing trends and periods, and processing the subsequences through normalization; secondly, introducing a large prediction model (LLM), screening events possibly influencing time sequence fluctuation from news data, and integrating related insight into time sequence analysis to enable the model to dynamically adapt to external events; and finally, constructing a long short-term memory (LSTM) network based on a self-attention mechanism, mapping the output of the LSTM network to a target output space through a full connection layer, and generating a final prediction result. According to the method, the advantages of a statistical method and a deep learning model are combined, the precision of complex time sequence prediction is improved, and especially under the influence of a dynamic environment and emergencies, the fluctuation change of the time sequence can be more accurately captured.

Description

Technical Field

[0001] The present invention belongs to the field of time series prediction, and in particular to a time series prediction method based on a deep learning model. Background Art

[0002] Time series forecasting plays an important role in resource management, decision support, risk control, and customer experience improvement. Accurate forecasting can provide strategic advantages for organizations, improve responsiveness and adaptability, and promote sustainable development. In an ever-changing environment, the significance of time series forecasting becomes increasingly prominent.

[0003] At present, the methods of time series prediction mainly include machine learning algorithms based on statistics and deep learning models. Using machine learning algorithms based on statistics to analyze time series is one of the important technologies for realizing time series prediction. It mainly analyzes historical medical data through the autoregressive integrated moving average model (ARIMA) and seasonal decomposition (STL), identifies trends, seasonality and cyclical changes, and realizes time series prediction. These models perform well when processing stationary data and can provide a basic reference for short-term prediction. Deep learning models have many advantages in time series prediction, including capturing complex nonlinear relationships, processing large-scale data, automatically learning features, capturing long-term dependencies, etc. These advantages make deep learning models an important method for realizing time series prediction.

[0004] However, traditional statistical models often have difficulty capturing the complex nonlinear relationships between variables, and time series analysis usually requires stability tests and seasonal adjustments, which makes the model building process relatively complicated. In addition, most of the existing time series prediction methods based on deep learning use models such as long short-term memory networks (LSTM) and gated recurrent units (GRU) to process sequence data, which can effectively capture long-term dependencies in the data, but cannot handle the fluctuations caused by sudden events. Summary of the invention

[0005] The purpose of the present invention is to propose a time series prediction method based on a deep learning model, which uses the statistical method variational mode decomposition (VMD) to split the time series signal, combines the large prediction model, and integrates the fluctuation impact caused by sudden events to solve the technical problem of accurately predicting complex time series.

[0006] To achieve the above purpose, the following steps are included:

[0007] Step S1: Data preprocessing, processing missing values ​​and outliers in time series data, and extracting timestamps;

[0008] Step S2: using a variational mode decomposition (VMD) model, the time series data is decomposed into multiple time series signals, and the time series signals are normalized;

[0009] Step S3: Using the large prediction model (LLM), we filter news about breaking events that may cause series fluctuations and incorporate news insights into the time series;

[0010] Step S4: Building a long short-term memory (LSTM) time series prediction neural network based on self-attention mechanism;

[0011] Step S5: Map the output of the long short-term memory network (LSTM) to the target output space through a fully connected layer as the prediction result;

[0012] Step S6: After training the constructed deep learning neural network and obtaining the prediction model, the time series data to be predicted is input into the prediction model and the corresponding prediction results are output.

[0013] Specifically, in step S1, for the input time series data X = {x1, x2, ..., x n}, using linear interpolation to handle missing values, using standard scores Identify outliers, where μ is the mean and σ is the standard deviation, and replace them with the mean of the previous and next data.

[0014] Furthermore, in step S2, the complex time series is decomposed into several sequences of different frequencies through the variational mode decomposition (VMD) model, so that each modal signal has a finite bandwidth in the frequency domain:

[0015]

[0016] Among them, u k (t) is each mode, ω k is the center frequency of each mode, and j is an imaginary unit. After being decomposed into multiple time series signals, they are normalized separately:

[0017]

[0018] Among them, x t is the decomposed time series data, x min is the minimum value, x max is the maximum value.

[0019] Furthermore, in step S3, first, the news text of emergencies that may cause sequence fluctuations is preprocessed, mainly including steps such as removing special characters and tags, converting to lowercase, removing stop words and punctuation marks, and using a tokenizer to decompose the text into subword units. Secondly, the tokens after tokenization are converted into vocabulary indices, and the corresponding attention encodings are generated at the same time. The vocabulary indices and attention encodings are processed using the BERT model, and a tensor HiddenStates = (BatchSize, SeqLen, HiddenSize) is output, where SeqLen is the input sequence length, and the last layer is extracted as the feature vector FetureVector; finally, the semantic similarity is calculated using cosine similarity:

[0020]

[0021] Among them, A and B are the feature vectors of text and keywords respectively, and the similarity calculation formula is used to calculate the news text matching score:

[0022]

[0023] Among them, FeatureVector is the feature vector of the news text, Keyword i is the feature vector of the keyword, and the news text is filtered through the semantic matching score.

[0024] Furthermore, the filtered text feature vector E(T t ) is mapped to the time series feature vector x t Same dimensions:

[0025] E'(T t )=W e ·E(T t )+b e

[0026] Among them, W e is the weight matrix of linear transformation, b e As the bias vector, the transformed text feature vector E'(T t ) and the time series feature vector x t To perform the fusion:

[0027] Z t =E'(T t )+x t

[0028] Furthermore, in step S4, the input sequence is processed using the self-attention mechanism, and the input feature sequence Z∈R after incorporating news insights is processed through three different weight matrices. n×dTransformed into query, key and value:

[0029] Q=ZW Q ,K=ZW K ,V=ZW V

[0030] Among them, W Q , W K , W V is the learned weight matrix with a shape of d×d. Then calculate the dot product of the query and the key to get the attention score, and then normalize it through the Softmax function to get the attention weight:

[0031]

[0032] Among them, d k is the dimension of the key.

[0033] Furthermore, the attention weight is combined with the weighted sum of the multi-feature time series X∈R n×d Input LSTM model. Each time step corresponds to an LSTM unit. Each LSTM unit consists of three parts: forget gate, input gate and output gate, and stores the state of long-term information through cell state. First, the forget gate f t Controls how much information to forget from the cell state:

[0034] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0035] Among them, σ is the sigmoid activation function, W f is the weight matrix, h t-1 is the hidden state of the previous time step, x t is the input feature vector of the current time step, b f is the bias term. After passing through the forget gate, the input gate i t Control the impact of current input information on cell state:

[0036] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0037] Among them, W i is the weight matrix, b i is the bias term. After passing through the input gate, the candidate cell state is updated And according to the candidate cell state, update the cell state Ct :

[0038]

[0039] Among them, tanh is the hyperbolic tangent activation function, W c is the weight matrix, b c is the bias term. Finally, it passes through the output gate o t , and update the hidden state h t :

[0040] o t =σ(W o ·[h t-1 ,x t +b o ])

[0041] h t =o t *tanh(C t )

[0042] Among them, W o is the weight matrix, b o is the bias term.

[0043] Further, in step S5, the hidden state output of LSTM is H = [h1,h2,...,h T ], through the fully connected layer, mapped to the target output space as the prediction result:

[0044] y=f(W·H+b)

[0045] Among them, f is the activation function, W is the weight matrix, and b is the bias term.

[0046] Furthermore, in step S6, the constructed deep learning neural network is trained, and hyperparameters such as the number of LSTM hidden units and the learning rate are reasonably adjusted to optimize the prediction model. Finally, the time series data to be predicted is input into the prediction model, and the corresponding prediction results are output.

[0047] Compared with the prior art, the present invention has the following advantages:

[0048] The present invention firstly uses the variational mode decomposition (VMD) model time series signal processing method to effectively decompose the complex time series into several intrinsic mode functions, which can extract the main components in the signal, remove noise, and improve the accuracy of model prediction; then uses the large prediction model (LLM) to screen the news of sudden events that may cause sequence fluctuations, and integrates news insights into the time series to deal with the fluctuation impact caused by sudden events; finally, the self-attention mechanism is used to effectively capture long-term dependencies, avoid the gradient vanishing problem, and enhance the interpretability of the model, thereby improving the accuracy and efficiency of prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flowchart of the steps of a time series prediction method based on a deep learning model according to the technical solution of the present invention;

[0050] Figure 2 This is an overall framework diagram of a time series prediction method based on a deep learning model according to the technical solution of the present invention;

[0051] Figure 3 It is a modal decomposition diagram of a time series prediction method based on a deep learning model according to the technical solution of the present invention;

[0052] Figure 4 It is an output fusion graph of a time series prediction method based on a deep learning model according to the technical solution of the present invention; DETAILED DESCRIPTION

[0053] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings.

[0054] like Figure 1 As shown, a time series prediction method based on a deep learning model includes the following steps:

[0055] Step S1: Data preprocessing, processing missing values ​​and outliers in time series data, and extracting timestamps.

[0056] For the input time series data X={x1,x2,...,x n}, using linear interpolation to handle missing values, using standard scores Identify outliers, where μ is the mean and σ is the standard deviation, and replace them with the mean of the previous and next data.

[0057] Step S2: Using the variational mode decomposition (VMD) model, the complex time series is decomposed into several sequences of different frequencies, so that each modal signal has a limited bandwidth in the frequency domain:

[0058]

[0059] Among them, u k (t) is each mode, ω k is the center frequency of each mode, and j is an imaginary unit. After being decomposed into multiple time series signals, they are normalized separately:

[0060]

[0061] Among them, x t is the decomposed time series data, x minis the minimum value, x max is the maximum value.

[0062] Step S3: Using the large prediction model (LLM), we filter news about breaking events that may cause series fluctuations and integrate news insights into the time series.

[0063] First, the news text of emergencies that may cause sequence fluctuations is preprocessed, mainly including removing special characters and tags, converting to lowercase, removing stop words and punctuation, and using a tokenizer to decompose the text into subword units. Secondly, the tokens after tokenization are converted into vocabulary indices, and the corresponding attention encodings are generated at the same time. The BERT model is used to process the vocabulary indices and attention encodings, and a tensor HiddenStates = (BatchSize, SeqLen, HiddenSize) is output, where SeqLen is the input sequence length, and the last layer is extracted as the feature vector FetureVector; finally, the cosine similarity is used to calculate the semantic similarity:

[0064]

[0065] Among them, A and B are the feature vectors of text and keywords respectively, and the similarity calculation formula is used to calculate the news text matching score:

[0066]

[0067] Among them, FeatureVector is the feature vector of the news text, Keyword i is the feature vector of the keyword, and the news text is filtered through the semantic matching score.

[0068] Furthermore, the filtered text feature vector E(T t ) is mapped to the time series feature vector x t Same dimensions:

[0069] E'(T t )=W e ·E(T t )+b e

[0070] Among them, W e is the weight matrix of linear transformation, b e As the bias vector, the transformed text feature vector E'(T t ) and the time series feature vector x t To perform the fusion:

[0071] Z t =E'(T t )+xt

[0072] Step S4: Use the self-attention mechanism to process the input sequence, and use three different weight matrices to integrate the input feature sequence Z∈R n×d Transformed into query, key and value:

[0073] Q=ZW Q ,K=ZW K ,V=ZW V

[0074] Among them, W Q , W K , W V is the learned weight matrix with a shape of d×d. Then calculate the dot product of the query and the key to get the attention score, and then normalize it through the Softmax function to get the attention weight:

[0075]

[0076] Among them, d k is the dimension of the key.

[0077] Furthermore, the attention weight is combined with the weighted sum of the multi-feature time series X∈R n×d Input LSTM model. Each time step corresponds to an LSTM unit. Each LSTM unit consists of three parts: forget gate, input gate and output gate, and stores the state of long-term information through cell state. First, the forget gate f t Controls how much information to forget from the cell state:

[0078] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0079] Among them, σ is the sigmoid activation function, W f is the weight matrix, h t-1 is the hidden state of the previous time step, x t is the input feature vector of the current time step, b f is the bias term. After passing through the forget gate, the input gate i t Control the impact of current input information on cell state:

[0080] i t =σ(W i ·[h t-1 ,x t ]+b i )

[0081] Among them, W i is the weight matrix, b i is the bias term. After passing through the input gate, the candidate cell state is updated And according to the candidate cell state, update the cell state C t :

[0082]

[0083] Among them, tanh is the hyperbolic tangent activation function, W c is the weight matrix, b c is the bias term. Finally, it passes through the output gate o t , and update the hidden state h t :

[0084] o t =σ(W o ·[h t-1 ,x t +b o ])

[0085] h t =o t *tanh(C t )

[0086] Among them, W o is the weight matrix, b o is the bias term.

[0087] Step S5: Output the hidden state of the last unit of LSTM H = [h1,h2,...,h T ], through the fully connected layer, mapped to the target output space as the prediction result:

[0088] y=f(W·H+b)

[0089] Among them, f is the activation function, W is the weight matrix, and b is the bias term.

[0090] Step S6: Train the constructed deep learning neural network, and reasonably adjust the hyperparameters such as the number of LSTM hidden units and learning rate to optimize the prediction model. Finally, input the time series data to be predicted into the prediction model and output the corresponding prediction results.

[0091] The above description is only a specific implementation of the present invention, but the protection scope of the present invention is not limited thereto. Any person familiar with the technology can understand and think of any changes or substitutions within the technical scope disclosed by the present invention, which should be included in the scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A time series prediction method based on a deep learning model, comprising the following steps: Step S1: Data preprocessing, processing missing values ​​and outliers in time series data, and extracting timestamps; Step S2: using a variational mode decomposition (VMD) model, the time series data is decomposed into multiple time series signals, and the time series signals are normalized; Step S3: Using the large prediction model (LLM), we filter news about breaking events that may cause series fluctuations and incorporate news insights into the time series; Step S4: Building a long short-term memory (LSTM) time series prediction neural network based on self-attention mechanism; Step S5: Map the output of the long short-term memory network (LSTM) to the target output space through a fully connected layer as the prediction result; Step S6: After training the constructed deep learning neural network and obtaining the prediction model, the time series data to be predicted is input into the prediction model and the corresponding prediction results are output.

2. The time series prediction method based on deep learning model according to claim 1 is characterized in that: Process missing values ​​and outliers in the input time series data and extract timestamps. Step S1: For the input time series data X = {x1, x2, ..., x n }, using linear interpolation to handle missing values, using standard scores Identify outliers, where μ is the mean and σ is the standard deviation, and replace them with the mean of the previous and next data.

3. The time series prediction method based on deep learning model according to claim 1 is characterized in that: In step S2, the time series data is decomposed into multiple time series signals, and the time series signals are normalized so that each modal signal has a finite bandwidth in the frequency domain: Among them, u k (t) is each mode, ω k is the center frequency of each mode, and j is an imaginary unit. After being decomposed into multiple time series signals, they are normalized separately: Among them, x t is the decomposed time series data, x min is the minimum value, x max is the maximum value.

4. The time series prediction method based on deep learning model according to claim 1, characterized in that: In step S3, a large prediction model (LLM) is used to screen news of emergencies that may cause sequence fluctuations, and to integrate news insights into the time series. Step S3: First, preprocess the news text of emergencies that may cause sequence fluctuations, mainly including removing special characters and tags, converting to lowercase, removing stop words and punctuation, and using a tokenizer to decompose the text into subword units. Secondly, convert the tokens after tokenization into vocabulary indexes, and generate corresponding attention codes at the same time. Use the BERT model to process the vocabulary index and attention code, and output a tensor HiddenStates = (BatchSize, SeqLen, HiddenSize), where SeqLen is the input sequence length, and extract the last layer as the feature vector FetureVector; finally, use cosine similarity to calculate semantic similarity: Among them, A and B are the feature vectors of text and keywords respectively, and the similarity calculation formula is used to calculate the news text matching score: Among them, FeatureVector is the feature vector of the news text, Keyword i is the feature vector of the keyword, and the news text is filtered through the semantic matching score. Furthermore, the filtered text feature vector E(T t ) is mapped to the time series feature vector x t Same dimensions: E'(T t )=W e ·E(T t )+b e Among them, W e is the weight matrix of linear transformation, b e As the bias vector, the transformed text feature vector E'(T t ) and the time series feature vector x t To perform the fusion: Z t =E'(T t )+x t 。 5. The time series prediction method based on deep learning model according to claim 1, characterized in that: The step S4 is to build a long short-term memory (LSTM) time series prediction neural network based on the self-attention mechanism; Step S4: Use the self-attention mechanism to process the input sequence, and use three different weight matrices to integrate the input feature sequence Z∈R n×d Transformed into query, key and value: Q=ZW Q ,K=ZW K ,V=ZW V Among them, W Q , W K , W V is the learned weight matrix with a shape of d×d. Then calculate the dot product of the query and the key to get the attention score, and then normalize it through the Softmax function to get the attention weight: Among them, d k is the dimension of the key. Furthermore, the attention weight is combined with the weighted sum of the multi-feature time series X∈R n×d Input LSTM model. Each time step corresponds to an LSTM unit. Each LSTM unit consists of three parts: forget gate, input gate and output gate, and stores the state of long-term information through cell state. First, the forget gate f t Controls how much information to forget from the cell state: f t =σ(W f ·[h t-1 ,x t ]+b f ) Among them, σ is the sigmoid activation function, W f is the weight matrix, h t-1 is the hidden state of the previous time step, x t is the input feature vector of the current time step, b f is the bias term. After passing through the forget gate, the input gate i t Control the impact of current input information on cell state: i t =σ(W i ·[h t-1 ,x t ]+b i )W i is the weight matrix, b i is the bias term. After passing through the input gate, the candidate cell state is updated And according to the candidate cell state, update the cell state C t : Among them, tanh is the hyperbolic tangent activation function, W c is the weight matrix, b c is the bias term. Finally, it passes through the output gate o t , and update the hidden state h t : the t =σ(W o ·[h t-1 ,x t +b o ]) h t =o t *tanh(C t ) Among them, W o is the weight matrix, b o is the bias term.

6. The time series prediction method based on deep learning model according to claim 1, characterized in that: In step S5, the hidden state output of LSTM is H=[h1,h2,...,h T ], through the fully connected layer, mapped to the target output space as the prediction result: y=f(W·H+b) Among them, f is the activation function, W is the weight matrix, and b is the bias term.

7. The time series prediction method based on deep learning model according to claim 1, characterized in that: The step S6 further trains the constructed deep learning neural network, and after obtaining the prediction model, the time series data to be predicted is input into the prediction model, and the corresponding prediction results are output.

Images