A method, device and computer equipment for predicting time series data based on CGA fusion model
Through the CGA fusion model, combined with multi-scale convolutional neural network and gated recurrent unit network, and combined with autoregressive model, the problem of failure to consider time scale, local and global features in the existing technology is solved, and a higher precision is achieved. Time series prediction.
Patent Information
- Application Number
- CN202210169309.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-23
- Publication Date
- 2024-12-20
- Estimated Expiration
- 2042-02-23
AI Technical Summary
The existing time series prediction method based on neural networks fails to consider the local characteristics of time series data on different time scales, the global timing characteristics of data, and the burst characteristics of time series data, resulting in low prediction accuracy and limited performance of a single model. Applicable scenarios are limited.
The time series data prediction method based on CGA fusion model is adopted, including the first layer multi-channel multi-scale convolutional neural network, the gated recurrent unit network, the second layer multi-channel multi-scale convolutional neural network and the autoregressive model, through multi-layer Structure extracts nonlinear and linear features, combining stacking and parallel methods to reduce the difficulty of model training.
It improves the richness of feature extraction and prediction accuracy, can more accurately capture local features and global timing features of multiple time scales, reduces the difficulty of model training, avoids information loss, and enhances the accuracy of prediction results.
Smart Images
Figure CN114565149B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of time series prediction, and specifically to a time series data prediction method, device and computer equipment based on a CGA fusion model. Background Art
[0002] Time series prediction tasks are a common type of tasks in machine learning. They are widely used in finance, industry, manufacturing, transportation and other fields, such as power load prediction, traffic flow prediction, stock price prediction, weather data prediction, etc. In recent years, with the rapid development of deep learning technology, deep learning technology has been proven to be a very important modeling method in many fields. At the same time, some time series prediction methods have also been developed. However, there are still some problems with the existing time series prediction methods based on neural networks. For example, they do not consider the local characteristics of time series data at different time scales, the global time series characteristics of the data as a whole, and the burst characteristics of time series data at the same time, that is, they do not consider nonlinear characteristics and linear characteristics at the same time; or the performance of a single model is limited and the applicable scenarios are limited, resulting in low prediction accuracy. The following will describe the existing time series prediction methods.
[0003] A patent with application number CN202010496285.8 and titled "Multi-layer spatiotemporal dependency modeling method for multi-layer time series based on deep learning" discloses a multi-layer spatiotemporal dependency modeling method for multi-layer time series based on deep learning. This method mainly uses the CNN-LSTM structure to initially extract features, and is composed of CNN's spatial attention mechanism, CNN's channel attention mechanism, temporal attention mechanism, and autoregressive components. It introduces the concept of multi-layer spatiotemporal dependency, uses CNN-based channel attention mechanism and CNN-based spatial attention mechanism to focus on the spatiotemporal dependency features of different layers respectively, realizes the filtering of redundant information and the effective extraction of features that have a greater impact on the prediction results, and achieves the purpose of improving the prediction results. Although this method combines nonlinear features and linear features at the same time, it does not take into account the local features of different time scales and the stacked CNN-LSTM structure, which increases the difficulty of training the model.
[0004] The patent with application number CN201910460741.0 and titled "Time Series Prediction Method and Model Determination Method Based on CNN" discloses a time series prediction method and model determination method based on CNN. The method is: obtain historical time series data, determine the periodic parameters according to the periodic characteristics of the historical time series data, and the periodic parameters include the periodic type and its corresponding periodic parameters; determine the component data corresponding to the predicted time point in the historical time series data based on the predicted time point, historical time series data, periodic parameters and preset cycle span, and the component data includes the closest time period data and periodic data; use the determined CNN model to predict the component data to obtain the prediction result corresponding to the predicted time point. The core of this method is to use the periodic parameters of the data, extract features with CNN through long, short cycles and adjacent data respectively, and fuse them for prediction. Although this method takes into account the characteristics of different time scales, it has certain requirements for the periodicity of the data, and does not take into account the non-periodic time series dependency characteristics and short-term linear characteristics, which has certain limitations in prediction effect and applicable scenarios. Summary of the invention
[0005] In view of the shortcomings of the prior art, the present invention proposes a time series data prediction method, device and computer equipment based on a CGA fusion model. The method comprises: obtaining historical data of target data, preprocessing the historical target data, and obtaining preprocessed historical target data; the target data is traffic flow data; inputting the preprocessed historical target data into a constructed CGA fusion model to obtain a time series prediction result of the target data, and controlling the future traffic flow according to the time series prediction result; the CGA fusion model comprises a first layer of multi-channel multi-scale convolutional neural network, a gated recurrent unit network, a second layer of multi-channel multi-scale convolutional neural network and an autoregressive model;
[0006] The process of building a CGA fusion model includes:
[0007] S1: Input the historical target data into the first layer of multi-channel multi-scale convolutional neural network MCMSCNN to obtain primary features;
[0008] S2: Input the primary features into the gated recurrent unit network GRU to obtain the temporal features;
[0009] S3: Input the primary features into the second layer of multi-channel multi-scale convolutional neural network MCMSCNN to obtain local features;
[0010] S4: Input the historical target data into the autoregressive model AR to obtain linear features;
[0011] S5: Add the temporal features, local features and linear features to obtain the constructed CGA fusion model.
[0012] Preferably, the process of preprocessing the historical target data includes:
[0013] Remove outliers and fill missing values from historical target data to obtain complete target data;
[0014] Normalizing the perfect target data to obtain normalized data;
[0015] The sliding window length T is set, and the normalized data is sampled using the sliding window to obtain the preprocessed historical target data.
[0016] Preferably, the process of obtaining the primary features includes: using three convolution kernels of different sizes to perform convolution, activation and dropout processing on the input data to obtain feature tensors of three different lengths, namely the primary features.
[0017] Preferably, the process of obtaining the time series features includes: processing the primary features, inputting the processed primary features into a gated recurrent unit network GRU to obtain the time series features; processing the primary features includes:
[0018] Fill the two shorter feature tensors of three feature tensors of different lengths to obtain three feature tensors of the same length;
[0019] The three feature tensors of the same length are concatenated to obtain the processed primary features.
[0020] Preferably, the process of obtaining the local features includes:
[0021] Three convolution kernels of different sizes are used to perform convolution, activation, maximum pooling and dropout on the input data to obtain feature tensors of three different lengths;
[0022] The feature tensors of three different lengths are concatenated and flattened and output through a linear layer to obtain local features;
[0023] Preferably, the process of obtaining the time series prediction result of the target data includes:
[0024] Input the first sliding window sampling data into the CGA fusion model to obtain the predicted value at the current moment;
[0025] The target data is spliced with the predicted value at the current moment, the sliding window is used, and the next sliding window sampling data is intercepted and input into the CGA fusion model to obtain the predicted value at the next moment; this process is repeated to obtain the time series prediction result of the target data.
[0026] A time series data prediction device based on a CGA fusion model, comprising: a data preprocessing module, a primary feature extraction module, a local feature extraction module, a time series feature extraction module, a linear feature capture module and a time series prediction module;
[0027] The data preprocessing module is used to preprocess the historical target data;
[0028] The primary feature extraction module is used to extract features from the preprocessed historical target data and output primary features;
[0029] The local feature extraction module is used to extract local features based on the primary features and output local features;
[0030] The time series feature extraction module is used to obtain time series features according to the primary features;
[0031] The linear feature capture module is used to obtain linear features based on historical target data;
[0032] The time series prediction module is used to obtain a time series prediction result according to local features, time series features and linear features and output the time series prediction result.
[0033] A computer device comprises a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the above method are executed.
[0034] The beneficial effects of the present invention are as follows: compared with the existing time series prediction methods, the CGA fusion model proposed in the present invention comprehensively considers nonlinear features and linear features in terms of feature extraction, wherein the nonlinear features include local features of multiple time scales captured by MCMSCNN, and then the convolution output vectors of multiple scales are spliced, and on the one hand, they are sent to GRU to capture the time series features globally, and on the other hand, they are continuously sent to MCMSCNN with an added pooling layer to extract more complex local features; the AR model can capture short-time burst linear features, and finally obtain the prediction results. Since the features extracted from the time series data in the fusion model are rich, more accurate prediction results can be obtained. In terms of the fusion model, the present invention is different from the typical stacking and parallel types, but combines these two methods to fuse the MCMSCNN and GRU structures, and combines MCMSCNN, GRU and AR in a residual manner, which reduces the difficulty of model training and avoids the loss of original information caused by the convolution structure and the GRU structure, thereby further improving the prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 It is a schematic diagram of the CGA fusion model framework in the present invention;
[0036] Figure 2 A flow chart of constructing a CGA fusion model in the present invention;
[0037] Figure 3 Schematic diagram of multi-scale convolution in the present invention. DETAILED DESCRIPTION
[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0039] The present invention proposes a time series data prediction method, device and computer equipment based on CGA fusion model, such as Figure 1 As shown, the method includes: obtaining historical data of target data, preprocessing the historical target data, and obtaining preprocessed historical target data; the target data is traffic flow data; inputting the preprocessed historical target data into the constructed CGA fusion model to obtain the time series prediction result of the target data, and controlling the future traffic flow according to the time series prediction result; the CGA fusion model includes a first layer of multi-channel multi-scale convolutional neural network, a gated recurrent unit network, a second layer of multi-channel multi-scale convolutional neural network and an autoregressive model;
[0040] like Figure 2 As shown in Figure 2, the process of building a CGA fusion model includes:
[0041] S1: Input the historical target data into the first layer of multi-channel multi-scale convolutional neural network MCMSCNN to obtain primary features;
[0042] S2: Input the primary features into the gated recurrent unit network GRU to obtain the temporal features;
[0043] S3: Input the primary features into the second layer of multi-channel multi-scale convolutional neural network MCMSCNN to obtain local features;
[0044] S4: Input the historical target data into the autoregressive model AR to obtain linear features;
[0045] S5: Add the temporal features, local features and linear features to obtain the constructed CGA fusion model.
[0046] Acquire historical target data and preprocess the historical target data; specifically, remove outliers from the historical target data and fill in missing values to obtain complete target data; normalize the complete target data to obtain normalized data; set a sliding window length T, use a sliding window to sample the normalized data, and obtain preprocessed historical target data; preferably, the dimension of the historical target data can be multidimensional data, that is, not limited to unit or multivariate time series data.
[0047] like Figure 3 As shown, the historical target data is input into the first-layer multi-channel multi-scale convolutional neural network MCMSCNN to obtain primary features; the first-layer multi-channel multi-scale convolutional neural network MCMSCNN is composed of convolution kernels of multiple different sizes, preferably, the number of convolution kernels is 3, the sizes are 1×2, 1×3 and 1×4 respectively, and the number of convolution kernels of each size is greater than 2, for example, the number of convolution kernels of each size is 50; the first-layer MCMSCNN processes the historical target data as follows: convolution, activation and dropout are performed on the input data using convolution kernels of three different sizes to obtain feature tensors of three different lengths, and the feature tensors of the three different lengths are the primary features;
[0048] Processing of primary features:
[0049] The two shorter feature tensors of three feature tensors of different lengths are padded to obtain three feature tensors of the same length; the three feature tensors of the same length are concatenated to obtain processed primary features; the processed primary features are input into the gated recurrent unit network GRU to obtain time series features.
[0050] After the historical target data is subjected to preliminary local feature extraction by the first layer of MCMSCNN, it still has a temporal relationship in the time dimension; GRU is a recurrent neural network suitable for capturing temporal features. Compared with LSTM (long short-term memory network), its advantage is that it can have a smaller number of parameters under similar performance; preferably, the primary features are input into the gated recurrent unit network GRU corresponding to each time point to capture the temporal features. Optionally, in the gated recurrent unit neural network structure, the depth is not limited to one layer, and the gated recurrent unit structure can be replaced by other recurrent neural network structures (including but not limited to unidirectional and bidirectional RNN, LSTM, GRU, SRU and other network structures).
[0051] The primary features are input into the second-layer multi-channel multi-scale convolutional neural network MCMSCNN to obtain local features; the second-layer multi-channel multi-scale convolutional neural network MCMSCNN is composed of convolution kernels of various sizes, preferably, the number of convolution kernels is 3, the sizes are 1×2, 1×3 and 1×4 respectively, and the number of convolution kernels of each size is greater than 2, for example, the number of convolution kernels of each size is 50; the process of the second-layer MCMSCNN processing the primary features is:
[0052] Three convolution kernels of different sizes are used to perform convolution, activation, maximum pooling and dropout on the input data to obtain feature tensors of three different lengths;
[0053] The feature tensors of three different lengths are concatenated and flattened and output through a linear layer to obtain local features;
[0054] Preferably, the historical target data is input into the autoregressive model AR to obtain the linear feature:
[0055] feature AR =w T x+b
[0056] x=(x1,x2,...,x n )
[0057] Among them, feature AR represents linear features, x n It represents the value of the nth attribute of the historical target data, corresponding to the time series, which is the value of the nth moment in a training sample, w = (w1, w2, ..., w n ) and b are parameters that need to be trained.
[0058] Optionally, the regression model can be replaced by other linear models (including but not limited to ARIMA, ARMA, MA and other models).
[0059] Add the temporal features, local features and linear features to obtain the constructed CGA fusion model:
[0060] feature all =feature CNN +feature GRU +feature AR
[0061] Among them, feature CNN Represents the local features extracted by MCMSCNN (local features at multiple time scales); feature GRU Represents the temporal features extracted by GRU; featureAR Represents the linear features extracted by AR.
[0062] The preprocessed historical target data is input into the constructed CGA fusion model. The process of the CGA fusion model performing multi-step time series prediction based on the historical target data includes:
[0063] Use sliding window to intercept the first time window {t N ,…,t N+T-1}, input the sampled data into the CGA fusion model, and denormalize its output to obtain the current time t N+T The predicted value of
[0064] The target data is concatenated with the predicted value at the current moment, and the sliding window intercepts the next time window {t N+1 ,…,t N+T}, input the sampled data into the CGA fusion model, and obtain the next moment t N+T+1 Repeat this process to get the predicted value of the target data. N+T+2 ,t N+T+3 ,…} time series prediction results.
[0065] If only a single-step time series prediction is required, there is no need for a sliding window. The current sampling data is input into the CGA fusion model, and its output is denormalized to obtain the time series prediction result of the target data at the next moment.
[0066] Users can control future traffic flow based on time series prediction results.
[0067] The present invention also provides a time series data prediction device based on the CGA fusion model, comprising: a data preprocessing module, a primary feature extraction module, a local feature extraction module, a time series feature extraction module, a linear feature capture module and a time series prediction module;
[0068] The data preprocessing module is used to preprocess the historical target data;
[0069] The primary feature extraction module is used to extract features from the preprocessed historical target data and output primary features;
[0070] The local feature extraction module is used to extract local features based on the primary features and output local features;
[0071] The time series feature extraction module is used to obtain time series features according to the primary features;
[0072] The linear feature capture module is used to obtain linear features based on historical target data;
[0073] The time series prediction module is used to obtain a time series prediction result according to local features, time series features and linear features and output the time series prediction result.
[0074] The present invention also provides a computer device, comprising a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the above method are executed.
[0075] Compared with the existing time series prediction methods, the CGA fusion model proposed in the present invention comprehensively considers nonlinear features and linear features in feature extraction, wherein the nonlinear features include local features of multiple time scales captured by MCMSCNN, and then the convolution output vectors of multiple scales are spliced, and sent to GRU on the one hand to capture the time series features globally, and on the other hand, continue to be sent to MCMSCNN with an added pooling layer to extract more complex local features; the AR model can capture short-time burst linear features and finally obtain the prediction results. Since the features extracted from the time series data in the fusion model are rich, more accurate prediction results can be obtained. In terms of fusion models, the present invention is different from the typical stacking and parallel types, but combines the stacking and parallel methods to fuse the convolutional neural network and the gated recurrent unit neural network, and combines the autoregressive model in a residual manner to avoid the loss of original information caused by model stacking, reduce the difficulty of training, and further improve the prediction accuracy.
[0076] The above embodiments further illustrate the purpose, technical solutions and advantages of the present invention in detail. It should be understood that the above embodiments are only preferred implementation modes of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made to the present invention within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A time series data prediction method based on CGA fusion model, characterized in that: include: Acquire historical data of target data, preprocess the historical target data, and obtain preprocessed historical target data; The target data is traffic flow data; The preprocessed historical target data is input into the constructed CGA fusion model to obtain the time series prediction results of the target data, and the future traffic flow is controlled according to the time series prediction results; the CGA fusion model includes the first layer of multi-channel multi-scale convolutional neural network, gated recurrent unit network, the second layer of multi-channel multi-scale convolutional neural network and autoregressive model; The process of obtaining the time series prediction results of the target data includes: Input the first sliding window sampling data into the CGA fusion model to obtain the predicted value at the current moment; The target data is spliced with the predicted value at the current moment, the sliding window is used, and the next sliding window sampling data is intercepted and input into the CGA fusion model to obtain the predicted value at the next moment; this process is repeated to obtain the time series prediction result of the target data; The process of building a CGA fusion model includes: S1: Input the historical target data into the first layer of multi-channel multi-scale convolutional neural network MCMSCNN to obtain primary features; the process of obtaining primary features includes: using three different sizes of convolution kernels to perform convolution, activation and dropout processing on the input data to obtain three different lengths of feature tensors, namely primary features; S2: Input the primary features into the gated recurrent unit network GRU to obtain the time series features; the process of obtaining the time series features includes: processing the primary features, inputting the processed primary features into the gated recurrent unit network GRU to obtain the time series features; processing the primary features includes: Fill the two shorter feature tensors of three feature tensors of different lengths to obtain three feature tensors of the same length; The three feature tensors of the same length are concatenated to obtain the processed primary features; S3: Input the primary features into the second-layer multi-channel multi-scale convolutional neural network MCMSCNN to obtain local features. The process of obtaining local features includes: Three convolution kernels of different sizes are used to perform convolution, activation, maximum pooling and dropout on the input data to obtain feature tensors of three different lengths; The feature tensors of three different lengths are concatenated and flattened and output through a linear layer to obtain local features; S4: Input the historical target data into the autoregressive model AR to obtain linear features; S5: Add the temporal features, local features and linear features to obtain the constructed CGA fusion model.
2. The time series data prediction method based on the CGA fusion model according to claim 1 is characterized in that: The process of preprocessing historical target data includes: Remove outliers and fill missing values from historical target data to obtain complete target data; Normalizing the perfect target data to obtain normalized data; The sliding window length T is set, and the normalized data is sampled using the sliding window to obtain the preprocessed historical target data.
3. A time series data prediction device based on a CGA fusion model, the device being used to execute any one of the time series data prediction methods based on a CGA fusion model described in claims 1 to 2, characterized in that: include: Data preprocessing module, primary feature extraction module, local feature extraction module, time series feature extraction module, linear feature capture module and time series prediction module; The data preprocessing module is used to preprocess the historical target data; The primary feature extraction module is used to extract features from the preprocessed historical target data and output primary features; The local feature extraction module is used to extract local features based on the primary features and output local features; The time series feature extraction module is used to obtain time series features according to the primary features; The linear feature capture module is used to obtain linear features based on historical target data; The time series prediction module is used to obtain a time series prediction result according to local features, time series features and linear features and output the time series prediction result.
4. A computer device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the steps in the method according to any one of claims 1 to 2 are executed.