Time sequence prediction method based on long and short term characteristics

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a time series prediction method based on long and short term features, which comprises the following steps: inputting multi-dimensional time series data with the length of T; the method comprises the following steps: processing input time sequence data through a long and short feature layer in an encoder to generate long-term features and short-term features, and performing feature fusion on the extracted long-term features and short-term features; the long and short feature layer comprises a convolutional neural network and a long and short-term memory neural network, the convolutional neural network is used for extracting multi-dimensional data features of the input data, and the long and short-term memory neural network is used for extracting long-term and short-term features of the input data. By combining the long and short feature layers of the convolutional neural network and the long and short-term memory neural network, the short-term fluctuation and the long-term trend in the time sequence data can be effectively extracted, and the multi-scale feature extraction technology enables the model to better capture the complex time sequence dependency relationship in the data, so that the accuracy of time sequence prediction is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, in particular to the application of deep learning technology in time series prediction, specifically to a time series prediction method based on long-term and short-term features. Background Art

[0002] Time series prediction has important application value in the field of mining. Accurate time series prediction can help mining companies optimize resource allocation, improve production efficiency, reduce operational risks, and provide a scientific basis for decision-making. Through effective prediction of factors such as resource demand, equipment operating status and environmental changes in mining operations, companies can formulate operational strategies more reasonably and improve overall economic benefits.

[0003] Traditional time series forecasting methods usually rely on statistical models and rule-based methods. These methods can achieve certain results when facing simple linear data, but their prediction accuracy is often not ideal when dealing with complex mining data and high-dimensional data. The limitations of traditional models are mainly reflected in the following aspects: first, they cannot effectively capture the nonlinear relationship in the data, second, they lack the ability to handle long-term dependencies, and third, when facing a dynamically changing environment, the model has poor adaptability.

[0004] In recent years, the rapid development of deep learning technology has enabled neural network models to achieve remarkable results in various fields. Deep learning models have powerful automatic feature extraction capabilities and can learn complex patterns from large amounts of data. This ability is particularly important in mining prediction because this field involves complex interactions between multiple variables. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a time series prediction method based on long-term and short-term features, which solves the problems mentioned in the above background technology.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a time series prediction method based on long-term and short-term characteristics, comprising the following steps:

[0007] Input multidimensional time series data Y={y1,y2,…,yT} with length T;

[0008] The input time series data is processed through the long and short feature layers in the encoder to generate long-term features and short-term features, and the extracted long-term features and short-term features are fused.

[0009] Preferably, the long-short feature layer includes a convolutional neural network and a long short-term memory neural network, the convolutional neural network is used to extract the short-term change characteristics of the input data, and the long short-term memory neural network is used to extract the long-term trend characteristics of the input data. The output feature C of the convolution module is calculated by the following formula:

[0010] C = ReLU(Xt*Wc+bc);

[0011] Where Wc is the weight vector of the convolution kernel, bc is the offset, and * represents the convolution operation.

[0012] Preferably, the encoder includes a residual structure, which is used for cross-layer information transfer in long and short feature layers to alleviate the problem of gradient disappearance.

[0013] Preferably, the long and short feature layer in the encoder divides the output data of the convolutional neural network into short-term features x s , and the long-term characteristic x p , the short-term features are input into the LSTM network, and the long-term features are input into the L-LSTM network.

[0014] Preferably, the input gate, forget gate, and output gate of the LSTM network respectively control the input, forgetting, and output of short-term feature data, while the input gate, forget gate, and output gate of the L-LSTM network respectively control the input, forgetting, and output of long-term feature data; the LSTM structure processes short-term feature x s And L-LSTM structure to process long-term features x p , the calculation process of the LSTM part is:

[0015] i t =σ(W xi x s +W hi h t-1 +bi),

[0016] f t =σ(W xr x s +W hr h t-1 +b f ),

[0017] o t =σ(W xo x s +W ho h t-1 +b o ),

[0018] c′ t =tanh(W xc xs +W hc h t-1 +b c ),

[0019] c t =f t ☉c t-1 +i t ☉c′ t ,

[0020]

[0021] The calculation method of the L-LSTM part is the same as that of the LSTM part, except that the input data is the long-term feature x p .

[0022] Preferably, the outputs of the LSTM and L-LSTM in the recursive module are combined and input into the final output module, and a fully connected layer with F neurons is used to combine the outputs of the two, and the calculation formula is:

[0023]

[0024] Among them, W s , W p Represent the weight vector, b represents the offset value, where It refers to the final output of the entire long and short feature layer, representing the fusion of the long-term and short-term features of the input data, and its size is F×t.

[0025] Preferably, the output of the long-term and short-term feature layer will be converted through batch normalization operation and ReLU activation function, and the specific operation is:

[0026]

[0027] h” t =ReLU(h' t )=max(0,h' t );

[0028] Among them, h' t is the normalized data, μ h represents the mean of this batch of data, σ h Represents the sample variance, and on this basis, the data is stretched and offset (parameters γ and β), and the normalized data is passed through the activation function to obtain the result h" t .

[0029] Preferably, the decoder receives the output data h′ of the encoder and the future covariate Z t+w, and generate the time series prediction results through processing. The calculation formula is: D′=ReLU(W D ·(h′+z t+w )+b D ).

[0030] Preferably, the decoder comprises at least one residual block, which is used to integrate the output of the encoder with future covariates, and its output is obtained by residual operation: D′=ReLU(D+h′).

[0031] Preferably, the final output of the decoder is mapped to the result of time series prediction through a fully connected layer, and the calculation formula is: t+w =W y ·D′+b y .

[0032] The present invention provides a time series prediction method based on long-term and short-term characteristics. It has the following beneficial effects:

[0033] 1. The present invention can effectively extract short-term fluctuations and long-term trends in time series data by combining the long and short feature layers of convolutional neural networks and long short-term memory neural networks. This multi-scale feature extraction technology enables the model to better capture the complex temporal dependencies in the data, thereby significantly improving the accuracy of time series prediction.

[0034] 2. The present invention adopts batch normalization layer and ReLU activation function, which not only improves the training speed of the model, but also enhances the expressiveness of the model. Batch normalization improves the stability and generalization ability of the model, makes the training process more efficient, and reduces the risk of gradient disappearance, thereby making the training of deep networks more feasible.

[0035] 3. In the present invention, the encoder part realizes cross-layer information transmission through the residual structure, alleviates the gradient vanishing problem, and thus can capture longer-term temporal dependencies. The decoder part also adopts the residual module, which combines the output of the encoder and future covariates to effectively integrate historical observation features and future information, which not only improves the depth and expression ability of the model, but also enhances the ability to process complex time series data, thereby improving the prediction performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0037] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] Example:

[0039] Please refer to the attached Figure 1 The embodiment of the present invention provides a time series prediction method based on long-term and short-term characteristics, comprising the following steps:

[0040] Input multidimensional time series data Y={y1,y2,…,yT} with length T;

[0041] The input time series data is processed through the long and short feature layers in the encoder to generate long-term features and short-term features, and the extracted long-term features and short-term features are fused.

[0042] Specifically, the input data is processed through the long and short feature layer in the encoder to generate long-term features and short-term features, and these two types of features are effectively fused. The design of the long and short feature layer combines the convolutional neural network (CNN) and the long short-term memory neural network (LSTM). The convolutional neural network focuses on extracting the short-term change characteristics of the input data and can capture the rapidly changing patterns in the data, while the long short-term memory neural network is used to capture the long-term trend characteristics of the input data. This division of labor and cooperation effectively improves the performance of the model.

[0043] The long-short feature layer includes a convolutional neural network and a long short-term memory neural network. The convolutional neural network is used to extract the short-term change characteristics of the input data, and the long short-term memory neural network is used to extract the long-term trend characteristics of the input data. The output feature C of the convolution module is calculated by the following formula:

[0044] C = ReLU(Xt*Wc+bc);

[0045] Where Wc is the weight vector of the convolution kernel, bc is the offset, and * represents the convolution operation.

[0046] The encoder includes a residual structure, which is used to transfer information across layers in long and short feature layers to alleviate the problem of gradient disappearance.

[0047] Specifically, in the convolution module, the calculation formula of the output feature C is C=ReLU(Xt*Wc+bc), where Wc represents the weight vector of the convolution kernel, bc is the offset, and * represents the convolution operation. The convolution operation can effectively extract local features in time series data through the local perception mechanism, providing strong support for subsequent time series analysis. At the same time, in order to enhance the performance of the model, the encoder also introduces a residual structure, which aims to achieve efficient transmission of cross-layer information, thereby alleviating the problem of gradient disappearance, ensuring that information can flow effectively in deep networks, and improving the learning and generalization capabilities of the model.

[0048] The long and short feature layer in the encoder divides the output data of the convolutional neural network into short-term features x through the data segmentation module s , and the long-term characteristic x p , short-term features are input into the LSTM network, and long-term features are input into the L-LSTM network.

[0049] The input gate, forget gate, and output gate of the LSTM network control the input, forgetting, and output of short-term feature data respectively, while the input gate, forget gate, and output gate of the L-LSTM network control the input, forgetting, and output of long-term feature data respectively; the LSTM structure processes short-term features x s And L-LSTM structure to process long-term features x p , the calculation process of the LSTM part is:

[0050] i t =σ(W xi x s +W hi h t-1 +b i ),

[0051] f t =σ(W xf x s +W hf h t-1 +b f ),

[0052] o t =σ(W xo x s +W ho h t-1 +b o ),

[0053] c′ t =tanh(W xc x s +W hc h t-1 +b c ),

[0054] c t =f t ☉c t-1 +i t ☉c' t ,

[0055]

[0056] The calculation method of the L-LSTM part is the same as that of the LSTM part, except that the input data is the long-term feature x p .

[0057] Specifically, W xi , W xf , W xo , W xc Respectively refer to the weight vectors from the input layer to the input gate, forget gate, output gate and cell state; W hi , W hf , W ho , W hc Respectively refer to the weight vectors from the hidden layer to the input gate, forget gate, output gate and cell state; b i , b f , b o , b c Respectively refer to the offset of the input gate, forget gate, output gate and cell state; σ refers to the sigmoid activation function Tanh refers to the hyperbolic tangent activation function ⊙ represents the multiplication of vector elements. In the L-LSTM part that processes long-term features, the calculation method is the same as the above LSTM structure, except that the input data is converted into long-term features x p The number of neurons in the LSTM layer and the L-LSTM layer are L and S respectively, so their output and The sizes are: L×t and Specifically, the calculation process of the LSTM structure involves linear transformation and nonlinear activation of the input data in order to capture the dynamic changes in the time series and ensure that the model can adapt to the characteristics of different time scales.

[0058] The outputs of the LSTM and L-LSTM in the recursive module will be combined and input into the final output module. A fully connected layer with F neurons is used to combine the outputs of the two. The calculation formula is:

[0059]

[0060] Among them, W s , W p Represent the weight vector, b represents the offset value, where It refers to the final output of the entire long and short feature layer, representing the fusion of the long-term and short-term features of the input data, and its size is F×t.

[0061] Specifically, by weighted combination of the outputs of LSTM and L-LSTM, the model can simultaneously utilize the instantaneous changes reflected by short-term features and the trend changes revealed by long-term features. This feature fusion not only enhances the model's ability to understand time series data, but also improves its adaptability in the face of changing environments. In this way, the model can comprehensively consider short-term fluctuations and long-term trends when making predictions, ensuring the comprehensiveness and richness of information, thereby providing a solid foundation for subsequent predictions.

[0062] The output of the long-term and short-term feature layers will be transformed through batch normalization and ReLU activation function. The specific operation is:

[0063]

[0064] h″ t =ReLU(h′ t )=max(0,h' t );

[0065] Among them, h' t is the normalized data, μ h represents the mean of this batch of data, σ h Represents the sample variance, and on this basis, the data is stretched and offset (parameters γ and β), and the normalized data is passed through the activation function to obtain the result h″ t .

[0066] Specifically, it helps to improve the convergence speed of the model, and can also enhance the stability and anti-interference ability of the model. After the normalized data passes through the activation function, the final feature representation is generated, which provides a solid foundation for subsequent predictions.

[0067] The decoder receives the encoder’s output data h′ and the future covariate Z t+w , and generate the time series prediction results through processing. The calculation formula is: D = ReLU (W D ·(h′+z t+w )+b D ).

[0068] The decoder includes at least one residual block, which is used to integrate the output of the encoder with future covariates, and its output is obtained through a residual operation: D′=ReLU(D+h′).

[0069] The final output of the decoder is mapped to the result of time series prediction through the fully connected layer, and the calculation formula is: t+w =Wy ·D′+b y .

[0070] Specifically, the decoder part receives the output data and future covariates of the encoder, and generates time series prediction results through further processing. The design of the decoder includes at least one residual block, which aims to integrate the output of the encoder with future covariates in order to make full use of historical information and future trends. Its output is obtained through residual operations. Finally, the output of the decoder is mapped to the time series prediction result through a fully connected layer. Through the above steps, this method can effectively cope with complex time series data and meet the needs of practical applications.

[0071] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A time series prediction method based on long-term and short-term characteristics, characterized in that: The following steps are involved: Input multidimensional time series data Y={y1,y2,…,yT} with length T; The input time series data is processed through the long and short feature layers in the encoder to generate long-term features and short-term features, and the extracted long-term features and short-term features are fused.

2. The time series prediction method based on long-term and short-term characteristics according to claim 1 is characterized in that: The long-short feature layer includes a convolutional neural network and a long short-term memory neural network. The convolutional neural network is used to extract multidimensional data features of input data, and the long short-term memory neural network is used to extract long-term and short-term features of input data. The output feature C of the convolution module is calculated by the following formula: C = ReLU(Xt*Wc+bc); Where Wc is the weight vector of the convolution kernel, bc is the offset, and * represents the convolution operation.

3. The time series prediction method based on long-term and short-term characteristics according to claim 1 is characterized in that: The encoder includes a residual structure, which is used to transfer information across layers in long and short feature layers to alleviate the problem of gradient disappearance.

4. The time series prediction method based on long-term and short-term characteristics according to claim 1 is characterized in that: The long and short feature layer in the encoder divides the output data of the convolutional neural network into short-term features x through the data segmentation module s , and the long-term characteristic x p , the short-term features are input into the LSTM network, and the long-term features are input into the L-LSTM network.

5. The time series prediction method based on long-term and short-term characteristics according to claim 1 is characterized in that: The input gate, forget gate, and output gate of the LSTM network respectively control the input, forgetting, and output of short-term feature data, while the input gate, forget gate, and output gate of the L-LSTM network respectively control the input, forgetting, and output of long-term feature data; the LSTM structure processes short-term feature x s And L-LSTM structure to process long-term features x p , the calculation process of the LSTM part is: i t =σ(W xi x s +W hi h t-1 +b i ), f t =σ(W xf x s +W hf h t-1 +b f ), o t =σ(W xo x s +W ho h t-1 +b o ), c’ t =tanh(W xc x s +W hc h t-1 +b c ), c t =f t ⊙c t-1 +i t ⊙c’ t , The calculation method of the L-LSTM part is the same as that of the LSTM part, except that the input data is the long-term feature x p .

6. The time series prediction method based on long-term and short-term characteristics according to claim 1 is characterized in that: The outputs of LSTM and L-LSTM in the recursive module are combined and input to the final output module. A fully connected layer with F neurons is used to combine the outputs of the two. The calculation formula is: Among them, W s , W p Represent the weight vector, b represents the offset value, where It refers to the final output of the entire long and short feature layer, representing the fusion of the long-term and short-term features of the input data, and its size is F×t.

7. The time series prediction method based on long-term and short-term characteristics according to claim 1 is characterized in that: The output of the long-term and short-term feature layer will be transformed by batch normalization operation and ReLU activation function. The specific operation is: h” t =ReLU(h' t )=max(0,h' t ); Among them, h' t is the normalized data, μ h represents the mean of this batch of data, σ h Represents the sample variance, and on this basis, the data is stretched and offset (parameters γ and β), and the normalized data is passed through the activation function to obtain the result h" t .

8. The time series prediction method based on long-term and short-term characteristics according to claim 1 is characterized in that: The decoder receives the encoder output data h′ and the future covariate Z t+w , and generate time series prediction results through processing. The calculation formula is:

9. The time series prediction method based on long-term and short-term characteristics according to claim 1 is characterized in that: The decoder includes at least one residual block, which is used to integrate the output of the encoder with the future covariate, and its output is obtained by residual operation:

10. The time series prediction method based on long-term and short-term characteristics according to claim 1 is characterized in that: The final output of the decoder is mapped to the result of time series prediction through the fully connected layer, and the calculation formula is: t+w =W y ·D′+b y .

Images