Deep learning-based time series data prediction method

Abstract

The invention discloses a deep learning-based time series data prediction method, relates to the technical field of time series data prediction, and provides a new time series data analysis and prediction scheme, so that the prediction capability of the existing time series data is further improved. According to the method, feature extraction and extraction can be well performed on long-time sequence data, so that the overall prediction capability is improved, meanwhile, the model can have better catastrophic forgetting resistance than MLP through an improved full-connection layer, for data of a peak region in a one-dimensional regression task, KAN only reconstructs a region with data in a current stage, and the prediction efficiency is improved. And the previous region is kept unchanged, so that the problem of data remodeling of the whole region when the MLP faces new sample data is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of time series data prediction, and specifically to a time series data prediction method based on deep learning. Background Art

[0002] A time series is a sequence of data points arranged in chronological order. Usually, the time interval of a time series is a constant value (such as 1 second, 5 minutes, 12 hours, 7 days, 1 year), and the time series can be analyzed and processed as discrete time data. Time series are widely used in most applied sciences and engineering involving time data measurement, such as predictive analysis, anomaly detection, and sequence generation. In the field of deep learning, time series also have applications in speech recognition, video analysis, natural language processing, text sentiment analysis, and text generation. However, due to the large number of features and certain complexity of time series data, it is difficult to achieve accurate prediction.

[0003] Traditional econometric methods, such as autoregressive moving average (ARMA) and generalized autoregressive conditional heteroskedasticity (GARCH), have better prediction effects on data with less volatility. Traditional machine learning algorithms, such as random forests and support vector machines, are widely used as choices for regression prediction due to their simplicity and efficiency. Deep learning models have stronger learning and adaptive capabilities than traditional machine models, and can meet more flexible model building requirements. Deep learning is a collection of methods that imitate the way the human brain thinks. It is a branch of machine learning. Compared with machine learning, deep learning can extract deep features from data, so it can achieve better learning effects. In time series prediction, deep learning can more accurately identify data change patterns, trends, seasonal changes and other features, and make more accurate predictions.

[0004] Therefore, we need to study an efficient and accurate time series data analysis and prediction solution that can not only preprocess the data efficiently but also ensure a certain degree of accuracy of the analysis results. Summary of the invention

[0005] In view of the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a prediction method for time series data based on deep learning, which adds an attention mechanism and a novel KAN fully connected layer on the basis of the traditional CNN-LSTM, and has better prediction results than previous deep learning-based models.

[0006] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a prediction method for time series data based on deep learning, wherein the 1dCNN includes a one-dimensional convolutional layer, a ReLU activation function and a maximum pooling layer; the 1dCNN is used to extract local features of the sequence, and then the features are input into the BiLSTM layer for further long-sequence and long-time feature grasping, and then the parallel processing and multi-scale information capture of the attention mechanism are used to improve the model's recognition and attention to key features, and finally KAN is introduced to replace the MLP layer to generate the final output.

[0007] Preferably, the 1dCNN layer: this layer accepts preprocessed initial data, introduces nonlinearity through a ReLU activation function, so that the model can learn complex features from the data, alleviate the gradient vanishing problem, speed up the training, and reduce the spatial dimension of the data through a one-dimensional maximum pooling layer. The number of parameters is reduced to reduce the amount of calculation and prevent overfitting. This layer can be more adaptable to the processing of long sequence data.

[0008] Preferably, the BiLSTM layer: this layer accepts data processed by 1dCNN, and is mainly used to solve problems such as gradient disappearance or gradient explosion in the data processing process. It introduces a bidirectional mechanism based on the traditional LSTM model, which can fully retain the characteristic information of the data.

[0009] Preferably, the Attention layer: this layer accepts data processed by the BiLSTM layer, and is mainly used to enhance the model's attention to specific parts of the input data. It uses a multi-head attention mechanism to split the input into multiple "heads" for parallel processing. Each head learns a different representation of the input data, and then merges these representations to capture information in different subspaces, i.e., feature condensation.

[0010] Preferably, the KAN fully connected layer is based on the Kolmogorov-Arnold representation theorem and is implemented through the nonlinear activation function of two or more layers of Kolmogorov networks. After adding multiple layers of results, the required feature dimension conversion is achieved and the final result is output. KAN can learn and accurately represent complex functions more effectively than a single spline curve or MLP, and has wide application potential.

[0011] Compared with the prior art, the present invention has the following beneficial effects:

[0012] The present invention improves the traditional CNN-LSTM architecture and introduces an attention mechanism to better identify key features in the data, thereby improving the model's ability to perceive and process long sequence data, and improving the prediction accuracy of time series data and better performance;

[0013] The Kolmogorov-Arnold network replaces the traditional MLP as the fully connected layer, and effKAN is used for specific implementation. effKAN improves the efficiency by improving the fixed learnable activation function B-spline function of each layer in the network and replacing the L1 regularization of the weights, while retaining the advantages of the original KAN. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 Schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0015] See also Figure 1 , The present invention provides a technical solution: a prediction method for time series data based on deep learning, 1dCNN includes a one-dimensional convolutional layer, a ReLU activation function and a maximum pooling layer; 1dCNN is used to extract local features of the sequence, and then the features are input into the BiLSTM layer for further long-sequence and long-time feature grasping, and then the model is improved in recognition and attention to key features through parallel processing and multi-scale information capture by the attention mechanism, and finally KAN is introduced to replace the MLP layer to generate the final output, 1dCNN layer: This layer accepts the pre-processed initial data, introduces nonlinearity through a ReLU activation function, so that the model can learn complex features from the data, and alleviate the gradient disappearance problem, speed up the training speed, and reduce the spatial dimension of the data through a one-dimensional maximum pooling layer, reduce the number of parameters, so as to reduce the amount of calculation and prevent overfitting, and this layer can be more suitable for the processing of long sequence data, BiLSTM layer: This layer accepts data processed by 1dCNN, and is mainly used to solve the problems of gradient disappearance or gradient explosion in the data processing process. A bidirectional mechanism is introduced on the basis of the traditional LSTM model, which can fully retain the feature information of the data;

[0016] Attention layer: This layer accepts data processed by the BiLSTM layer and is mainly used to enhance the model's attention to specific parts of the input data. It uses a multi-head attention mechanism to split the input into multiple "heads" for parallel processing. Each head learns a different representation of the input data, and then merges these representations to capture information in different subspaces, i.e., feature condensation. KAN fully connected layer: Based on the Kolmogorov-Arnold representation theorem, it is implemented through the nonlinear activation function of more than two layers of Kolmogorov networks. After adding multiple layers of results, the required feature dimension conversion is achieved and the final result is output. KAN can learn and accurately represent complex functions more effectively than a single spline curve or MLP, and has the potential for wide application;

[0017] Based on the existing model, this paper proposes a new model that combines the attention mechanism and KAN. The main architecture of the model is CNN-BiLSTM-Attention-KAN. Feature extraction is based on one-dimensional convolutional neural network and bidirectional long short-term memory. The dual-headed attention mechanism is used for feature condensation and simplification. Finally, the relatively new Kolmogorov-Arnold network is used. The accuracy of time series training and prediction is improved through the integration of these deep learning frameworks.

Claims

1. A prediction method for time series data based on deep learning, characterized by: The 1dCNN includes a one-dimensional convolutional layer, a ReLU activation function and a maximum pooling layer; the 1dCNN is used to extract local features of the sequence, and then the features are input into the BiLSTM layer for further long sequence and long-time feature grasping, and then the parallel processing and multi-scale information capture of the attention mechanism are used to improve the model's recognition and attention to key features, and finally KAN is introduced to replace the MLP layer to generate the final output.

2. The method for predicting time series data based on deep learning according to claim 1, characterized in that: The 1dCNN layer: This layer accepts preprocessed initial data and introduces nonlinearity through a ReLU activation function, so that the model can learn complex features from the data, alleviate the gradient vanishing problem, speed up the training, and reduce the spatial dimension of the data through a one-dimensional maximum pooling layer. The number of parameters is reduced to reduce the amount of calculation and prevent overfitting. This layer can be more adaptable to the processing of long sequence data.

3. The method for predicting time series data based on deep learning according to claim 1, characterized in that: The BiLSTM layer: This layer accepts data processed by 1dCNN and is mainly used to solve problems such as gradient disappearance or gradient explosion in the data processing process. It introduces a bidirectional mechanism based on the traditional LSTM model, which can fully retain the feature information of the data.

4. The method for predicting time series data based on deep learning according to claim 1, characterized in that: The Attention layer: This layer accepts data processed by the BiLSTM layer and is mainly used to enhance the model's attention to specific parts of the input data. It uses a multi-head attention mechanism to split the input into multiple "heads" for parallel processing. Each head learns a different representation of the input data, and then merges these representations to capture information in different subspaces, i.e., feature condensation.

5. The method for predicting time series data based on deep learning according to claim 1, characterized in that: The KAN fully connected layer is based on the Kolmogorov-Arnold representation theorem and is implemented through the nonlinear activation function of two or more layers of Kolmogorov networks. After adding multiple layers of results, the required feature dimension conversion is achieved and the final result is output. KAN can learn and accurately represent complex functions more effectively than a single spline curve or MLP, and has the potential for wide application.

Images