A method to improve the accuracy of deep learning time series prediction models

Abstract

This invention provides a method for improving the accuracy of deep learning time series prediction models, comprising: obtaining time series data corresponding to a feature to be tested from a first dataset; constructing a second dataset based on the feature to be tested; inputting the second dataset into a convolutional neural network to obtain extracted features; concatenating the extracted features and the time series data corresponding to the feature to be tested using the torch.cat algorithm to obtain hybrid convolutional features; inputting the hybrid convolutional features into a bidirectional long short-term memory (LSSM) neural network to obtain an output vector; concatenating the output vector and the time series data corresponding to the feature to be tested using the torch.cat algorithm to obtain hybrid bidirectional LSSM features; and inputting the hybrid bidirectional LSSM features into a fully connected neural network to obtain the prediction result of the feature to be tested. The accuracy of prediction is improved by utilizing features multiple times.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and more particularly to a method for improving the accuracy of deep learning time series prediction models. Background Technology

[0002] In recent years, with the development of machine learning and deep learning technologies, more and more researchers have been conducting in-depth studies on time series prediction tasks. Prediction techniques have evolved from single deep neural network prediction models such as Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM), and Gated Recurrent Units (GRU) to hybrid deep learning models, resulting in hybrid models such as Convolutional-LSTM (CNN-LSTM), Convolutional-Gated Recurrent Units (CNN-GRU), and their variants such as Convolutional-Bidirectional LSTM (CNN-BILSTM) and Convolutional-Bidirectional Gated Recurrent Units (CNN-BIGRU). Hybrid models are more effective than single models in making more accurate predictions of future time series changes and trends. Currently, various methods or structures are being combined with hybrid models to improve prediction performance. Examples include combining Empirical Mode Decomposition (EMD) with hybrid models and incorporating attention mechanisms into hybrid structures. While these methods ensure accuracy in model prediction, the accuracy can still be improved.

[0003] In view of this, the present invention provides a method for improving the accuracy of deep learning time series prediction models. Summary of the Invention

[0004] To address the aforementioned technical problem of improving prediction accuracy, this invention provides a method for enhancing the accuracy of deep learning time series prediction models. This invention primarily utilizes convolutional neural networks and bidirectional long short-term memory neural networks to achieve basic predictions, and then improves the accuracy of predicting future features by repeatedly utilizing historical features.

[0005] The technical means employed in this invention are as follows:

[0006] This invention provides a method for improving the accuracy of deep learning time series prediction models, comprising:

[0007] Obtain a first dataset, which includes time-series data set corresponding to the features;

[0008] Obtain the time series data corresponding to the feature to be tested from the first dataset;

[0009] A second dataset is constructed based on the features to be tested;

[0010] The second dataset is input into a convolutional neural network to extract features;

[0011] The torch.cat algorithm is used to concatenate the extracted features and the time series data corresponding to the features to be tested to obtain hybrid convolutional features;

[0012] The hybrid convolutional features are input into a bidirectional long short-term memory neural network to obtain the output vector;

[0013] The output vector and the time series data corresponding to the feature to be tested are concatenated using the torch.cat algorithm to obtain a hybrid bidirectional long short memory feature.

[0014] The hybrid bidirectional long short-term memory features are input into a fully connected neural network to obtain the prediction results of the features to be tested.

[0015] Further, obtaining the first dataset includes:

[0016] Within a preset time period, feature data groups are acquired once at preset time intervals to obtain the first dataset;

[0017] The feature data group includes multiple numerical values ​​corresponding to the features;

[0018] Select one of the features, and arrange all the values ​​corresponding to the selected feature in chronological order to obtain the time series data corresponding to the selected feature.

[0019] Further, the second dataset is constructed based on the features to be tested, including:

[0020] In the first dataset, all features are correlated with the feature to be tested, and the features with a correlation value greater than 0.7 are selected as the correlated features.

[0021] Obtain the first to Nth relevant datasets whose correlation with the first dataset exceeds a threshold, wherein the first to Nth relevant datasets are obtained in the same way as the first dataset;

[0022] The second dataset is formed by selecting time series data corresponding to the relevant features from the first relevant dataset to the Nth relevant dataset.

[0023] Furthermore, before inputting the second dataset into the convolutional neural network, the process further includes:

[0024] The second dataset is preprocessed, including:

[0025] Check if the second dataset has missing values ​​and / or blank values;

[0026] If the missing values ​​and / or blank values ​​do not exist, the second dataset is processed using a normalization algorithm.

[0027] Furthermore, if the missing values ​​and / or blank values ​​exist, an interpolation algorithm is used to interpolate the missing values ​​and / or blank values;

[0028] The second dataset after interpolation is processed using a normalization algorithm.

[0029] Furthermore, the convolutional neural network, the bidirectional long short-term memory neural network, and the fully connected neural network are trained 100 times, with a training batch size of 128, an initial learning rate of 0.001, a sliding window size of a preset time interval × the number of relevant features, a loss function of MSE, and an optimizer of Adam.

[0030] Furthermore, the convolutional neural network is a one-dimensional convolutional neural network, comprising a first convolutional layer, a second convolutional layer, and a third convolutional layer connected in sequence. The first convolutional layer has 64 feature detectors, the second convolutional layer has 64 feature detectors, and the third convolutional layer has 32 feature detectors.

[0031] Furthermore, the bidirectional long short-term memory neural network includes a first layer and a second layer connected together, the first layer including 100 neurons and the second layer including 50 neurons.

[0032] Furthermore, the fully connected neural network includes a first fully connected layer, a second fully connected layer, and a third fully connected layer connected in sequence. The first fully connected layer includes 100 neural units, the second fully connected layer includes 50 neural units, and the third fully connected layer includes 50 neural units.

[0033] Compared with the prior art, the present invention has the following advantages:

[0034] 1. The present invention provides a method for improving the accuracy of a deep learning time series prediction model. It uses a three-layer one-dimensional convolutional neural network to extract important features from time series data, a two-layer bidirectional long short memory neural network to combine the extracted features and the time series data of the features to be tested to obtain an output vector, and a fully connected neural network to combine the output vector and the time series data corresponding to the features to be tested to output the prediction result. The original time series features are used multiple times to enhance the prediction accuracy.

[0035] 2. The present invention provides a method for improving the accuracy of a deep learning time series prediction model. By connecting the calculation results of a convolutional neural network with specific features and then inputting them into the next layer of the neural network for calculation, the effective features extracted by the convolutional neural network can be enhanced, thereby improving the accuracy and precision of the prediction.

[0036] 3. The method for improving the accuracy of deep learning time series prediction models provided by this invention has the advantages of simple structure, low precision error, and high accuracy. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart illustrating a method for improving the accuracy of a deep learning time series prediction model provided by the present invention.

[0039] Figure 2 This is a schematic diagram of a framework for a method to improve the accuracy of a deep learning time series prediction model provided by the present invention.

[0040] Figure 3 This is a comparison chart of the root mean square error between the method of this invention and the control method.

[0041] Figure 4 This is a comparison chart of the determination coefficients of the method of the present invention and the control method. Detailed Implementation

[0042] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0043] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0044] Combination Figure 1 and Figure 2 , Figure 1 This is a flowchart illustrating a method for improving the accuracy of a deep learning time series prediction model provided by the present invention. Figure 2 This is a schematic diagram illustrating a framework for a method to improve the accuracy of a deep learning time series prediction model provided by the present invention, used to illustrate a specific embodiment of the method for improving the accuracy of a deep learning time series prediction model provided by the present invention, including:

[0045] S1: Obtain the first dataset, which includes time series data corresponding to the features;

[0046] S2: Obtain the time series data corresponding to the feature to be tested from the first dataset;

[0047] S3: Construct a second dataset based on the features to be tested;

[0048] S4: Input the second dataset into the convolutional neural network to obtain the extracted features;

[0049] S5: The torch.cat algorithm is used to concatenate the time series data corresponding to the extracted features and the features to be tested to obtain hybrid convolutional features;

[0050] S6: Input the hybrid convolutional features into a bidirectional long short-term memory neural network to obtain the output vector;

[0051] S7: The torch.cat algorithm is used to concatenate the output vector and the time series data corresponding to the feature to be tested to obtain a hybrid bidirectional long short memory feature.

[0052] S8: Input the hybrid bidirectional long short-term memory features into a fully connected neural network to obtain the prediction results of the features to be tested.

[0053] It is understandable that convolutional neural networks are used to extract features from time series data, bidirectional long short-term memory neural networks combine the extracted features and the time series data of the features to be tested to obtain an output vector, and fully connected neural networks combine the output vector and the time series data corresponding to the features to be tested to output the prediction result. The time series data corresponding to the features to be tested are used multiple times to enhance the prediction accuracy.

[0054] In some alternative embodiments, reference continues to be made to... Figure 1 and Figure 2 Obtain the first dataset, including:

[0055] Within a preset time period, feature data groups are acquired once at preset time intervals to obtain the first dataset;

[0056] The feature data set includes numerical values ​​corresponding to multiple features;

[0057] Select a feature, and arrange all the values ​​corresponding to the selected feature in chronological order to obtain the time series data corresponding to the selected feature.

[0058] It is understood that the preset time period and time interval can be set according to actual needs, and this embodiment does not impose specific limitations on them. In this embodiment, if the PM feature is used for time series prediction... 2.5 For example, a characteristic data set can be obtained from historical air pollutant data of a specific air quality monitoring station published by the national air quality monitoring station. This characteristic data set includes PM2.5... 2.5 Value, PM 10 Taking the values ​​of SO2, NO2, O3, CO, and AQI as examples, the dataset and its features can be adjusted according to the needs of time series prediction, and this embodiment does not impose specific limitations on this.

[0059] In some alternative embodiments, reference continues to be made to... Figure 1 and Figure 2 A second dataset is constructed based on the features to be tested, including:

[0060] In the first dataset, all features are correlated with the feature to be tested, and features with a correlation value greater than 0.7 are selected as correlated features.

[0061] Obtain the first to Nth relevant datasets that are more than the threshold of relevance to the first dataset. The first to Nth relevant datasets are obtained in the same way as the first dataset.

[0062] The second dataset is constructed by selecting time series data corresponding to the relevant features from the first to the Nth relevant datasets.

[0063] Specifically, if the first dataset corresponds to a certain air quality monitoring station, find the first to Nth air quality monitoring stations whose correlation with that air quality monitoring station exceeds a threshold; following the method used to obtain the first dataset, find the historical air pollutant data corresponding to the first to Nth air quality monitoring stations among the national air quality monitoring stations to obtain the first to Nth related datasets. Select the time series data corresponding to the relevant features from the first, second, and third related datasets to form the second dataset.

[0064] The characteristics of time series prediction are PM 2.5 For example, using correlated datasets considers the spatial effects of pollutants, and using correlated datasets of highly correlated sites can account for spatial effects without introducing too many variables that could interfere with prediction performance. Furthermore, using correlated features in calculations can further improve the accuracy of deep learning time series prediction models.

[0065] In some alternative embodiments, reference continues to be made to... Figure 1 Before inputting the second dataset into the convolutional neural network, the following steps are also included:

[0066] The second dataset is preprocessed, including:

[0067] Check if the second dataset contains missing and / or blank values;

[0068] If there are no missing values ​​and / or blank values, the normalization algorithm is used to process the second dataset.

[0069] If missing values ​​and / or blank values ​​exist, an interpolation algorithm is used to impute the missing values ​​and / or blank values;

[0070] The second dataset after interpolation is processed using a normalization algorithm.

[0071] Understandably, the interpolation algorithm uses the k-nearest neighbor interpolation algorithm, but it is not limited to this. After processing with a normalization algorithm, the data in the dataset is scaled so that all data in the second dataset are within the range of 0 to 1, which facilitates subsequent calculations. Preferably, after obtaining the first dataset, it is also necessary to preprocess the data in the first dataset. The preprocessing process for the first dataset is the same as that for the second dataset.

[0072] In some optional embodiments, the convolutional neural network, bidirectional long short-term memory neural network, and fully connected neural network are trained 100 times, with a training batch size of 128, an initial learning rate of 0.001, a sliding window size of a preset time interval × the number of relevant features, a loss function of MSE, and an optimizer of Adam.

[0073] Understandably, taking a three-year period from January 1, 2020 to December 31, 2022, with a one-hour time interval, as an example, we obtain a total dataset of 26,280 records. We then use the first 80% of the dataset as the training set and the last 20% as the test set. The training set is used to train convolutional neural networks, bidirectional long short-term memory neural networks, and fully connected neural networks. Specifically, a sliding window size of 4×9 can be used.

[0074] The convolutional neural network is a one-dimensional convolutional neural network, consisting of a first convolutional layer, a second convolutional layer, and a third convolutional layer connected in sequence. The first convolutional layer has 64 feature detectors, the second convolutional layer has 64 feature detectors, and the third convolutional layer has 32 feature detectors. Each convolutional layer has a kernel of 3 with padding of 1. The activation function is the ReLU function. Each convolutional layer is followed by a max pooling layer of size 2.

[0075] The bidirectional long short-term memory neural network consists of a first layer and a second layer, with the first layer containing 100 neurons and the second layer containing 50 neurons.

[0076] A fully connected neural network consists of a first fully connected layer, a second fully connected layer, and a third fully connected layer connected in sequence. The first fully connected layer contains 100 neural units, the second fully connected layer contains 50 neural units, and the third fully connected layer contains 50 neural units.

[0077] In some alternative embodiments, refer to Figure 3 and Figure 4 , Figure 3 This is a comparison graph of the root mean square error between the method of the present invention and the control method. Figure 4 This is a comparison graph of the determination coefficients of the method of the present invention and the control method. (Refer to...) Figure 3 The method described in this invention, compared to the prediction method based on a convolutional-bidirectional long short-term memory (CNN-BILSTM) model, has a significantly lower root mean square error (RMSE) than the prediction method based on the CNN-BILSTM model. Figure 4 The determination coefficient (R) of the method provided by this invention 2 The coefficient of determination (R²) is higher than that of prediction methods based on convolutional-bidirectional long short-term memory hybrid models. 2 Therefore, the method for improving the accuracy of deep learning time series prediction models provided by this invention has lower errors and higher accuracy.

[0078] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0079] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for improving the accuracy of a deep learning time series prediction model, characterized in that, The method comprises the following steps: obtaining a first data set, wherein the first data set comprises time series data corresponding to features; Characteristic data sets are obtained from historical data of atmospheric pollutants of a certain air quality monitoring site, and the characteristic data sets include PM 2.5 values, PM 10 values, SO2 values, NO2 values, O3 values, CO values, and AQI values obtaining the time series data corresponding to the to-be-tested feature from the first data set; constructing a second data set according to the to-be-tested feature; the first data set corresponds to an air quality monitoring station, and the first air quality monitoring station to the Nth air quality monitoring station with a correlation exceeding a threshold value is found with reference to the air quality monitoring station; the first data set is obtained in the same way as the first data set, and the first related data set to the Nth related data set corresponding to the air quality monitoring station is obtained; selecting the three related data sets with the highest correlation from the first related data set to the Nth related data set as the first related data set, the second related data set, and the third related data set; selecting the time series data corresponding to the related features from the first related data set, the second related data set, and the third related data set to form the second data set; inputting the second data set into a convolutional neural network to obtain extracted features; the convolutional neural network is a one-dimensional convolutional neural network, which comprises a first convolutional layer, a second convolutional layer, and a third convolutional layer connected in sequence, the first convolutional layer has 64 feature detectors, the second convolutional layer has 64 feature detectors, and the third convolutional layer has 32 feature detectors; using a torch.cat algorithm to connect the extracted features and the time series data corresponding to the to-be-tested feature to obtain mixed convolutional features; inputting the mixed convolutional features into a bidirectional long short-term memory neural network to obtain an output vector; the bidirectional long short-term memory neural network comprises a first layer and a second layer connected in sequence, the first layer comprises 100 neurons, and the second layer comprises 50 neurons; using the torch.cat algorithm to connect the output vector and the time series data corresponding to the to-be-tested feature to obtain mixed bidirectional long short-term memory features; inputting the mixed bidirectional long short-term memory features into a fully connected neural network to obtain a prediction result of the to-be-tested feature.

2. The method for improving the accuracy of a deep learning time series prediction model according to claim 1, wherein, The first data set is obtained by: obtaining a feature data group every preset time interval within a preset time period to obtain the first data set; the feature data group comprises a plurality of numerical values corresponding to the features; selecting a feature, and arranging all numerical values corresponding to the selected feature in chronological order to obtain time series data corresponding to the selected feature.

3. The method of claim 2, wherein, The second data set is constructed according to the to-be-tested feature, comprising: all features in the first data set are respectively correlated with the to-be-tested feature, and the features with a correlation calculation value greater than 0.7 are selected as related features; obtaining the first related data set to the Nth related data set with a correlation exceeding a threshold value with the first data set, and the first related data set to the Nth related data set is obtained in the same way as the first data set. The time series data corresponding to the relevant features is selected from the first to the Nth relevant data sets to form the second data set.

4. The method for improving the accuracy of a deep learning time series prediction model according to claim 1, wherein, Before the second data set is input into the convolutional neural network, the method further comprises: The second data set is preprocessed, comprising: checking whether the second data set has missing values and / or blank values; if the missing values and / or the blank values are not present, the second data set is processed using a normalization algorithm.

5. The method of claim 4, wherein, if the missing values and / or the blank values are present, the missing values and / or the blank values are interpolated using an interpolation algorithm; the second data set after interpolation is processed using a normalization algorithm.

6. The method for improving the accuracy of a deep learning time series prediction model according to claim 1, wherein, The training rounds of the convolutional neural network, the bidirectional long short-term memory neural network and the fully connected neural network are 100, the training batch is 128, the initial learning rate is 0.001, the sliding window size is a preset time interval multiplied by the number of relevant features, the loss function is MSE, and the optimizer is Adam.

7. The method for improving the accuracy of a deep learning time series prediction model according to claim 1 or 2, characterized in that, The fully connected neural network comprises a first layer of full connection, a second layer of full connection and a third layer of full connection connected in sequence, the first layer of full connection comprises 100 neural units, the second layer of full connection comprises 50 neural units, and the third layer of full connection comprises 50 neural units.

Images