Time series prediction model training method based on data enhancement

Abstract

The invention provides a time series prediction model training method based on data enhancement. The method comprises the following steps: dividing real time series data into an original training set and a test set; the method comprises the following steps: constructing an MAD-TimeGAN network based on the TimeGAN network by adopting an MAD-GAN architecture, and generating sample time sequence data by adopting the MAD-TimeGAN network; combining the original training set with the sample time sequence data to obtain an enhanced training set; training the time sequence prediction model by adopting an enhanced training set; and evaluating a prediction result of the time sequence prediction model by adopting the test set. According to the method, the MAD-TimeGAN network is constructed based on the TimeGAN network by adopting the MAD-GAN architecture to generate more diversified sample time sequence data, and the sample time sequence data and the real time sequence data jointly form the training set to train the time sequence prediction model, so that the problem of data shortage in time sequence prediction is solved; and the performance of the time sequence prediction model and the quality of the generated sample are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a time series prediction model training method based on data enhancement. Background Art

[0002] In recent years, with the continuous development of computer technology and deep learning, time series prediction methods have been widely used in finance, meteorology, medicine, energy and other fields. Accurate time series prediction can provide scientific decision-making support for various industries. Generally speaking, time series prediction methods are extremely data-dependent prediction methods, and their prediction models often require a large amount of training set data to capture the complex characteristics of the data. However, in practical applications, data collection is difficult and costly, and the data sample size is small. In the absence of training data, the accuracy of the resulting prediction model will be difficult to achieve the expected goal. Therefore, the problem of time series data enhancement under small samples is of great research significance.

[0003] In small sample time series prediction, data enhancement technology is the most common method to improve the prediction accuracy of the model. Researchers use data enhancement technology to generate data similar to the original data distribution, so as to achieve the goal of expanding training data and improving the prediction accuracy of the model. The purpose of data enhancement is achieved by adding noise signals to the original signals to generate new synthetic data. The prior art proposes a virtual sample generation algorithm based on Gaussian Mixture Model (GMM). This method simulates the data distribution by mixing multiple Gaussian distributions to generate virtual samples with the same statistical characteristics. Some use variational auto encoders (VAE) to generate samples to solve the problem of missing samples in regression prediction. At the same time, the prior art also uses a virtual sample generation algorithm based on extreme learning machine (ELM). This method simulates the data distribution by quickly learning model parameters, and then generates data with similar characteristics. There is also a mixed data enhancement (Mixup) method, which realizes data expansion by linearly combining different samples and labels in proportion. Existing methods use generative adversarial networks to expand historical data to address the problem of lack of historical data in small sample load forecasting, which greatly alleviates the impact of data shortage on the forecasting model. However, the above data enhancement methods only consider the static characteristics of the data when generating data, and cannot reflect the dynamic information of the time series. Using the virtual samples generated by them cannot maximize the accuracy of time series forecasting.

[0004] To solve the above problems, further attention is paid to the dynamic information of data in the time dimension. A time series generation network TimeGAN is proposed based on the GAN network. The network jointly trains the autoencoder network and the generative adversarial network through three different loss functions to fully learn the dynamic information distribution of the time series and generate time series data with good quality. Some studies use TimeGAN to perform data augmentation on the GPS spoofing jamming signal of drones and use the prediction model for classification. Compared with the model without augmentation, the results show higher classification accuracy. The existing technology uses TimeGAN to perform data augmentation on battery degradation features to improve the prediction accuracy of the battery remaining life prediction model. However, although the TimeGAN model has been widely used in various fields, it still has the common mode collapse problem in the GAN network. TimeGAN realizes data augmentation of time series by jointly training the autoencoder network and the adversarial network. However, during the TimeGAN training process, if the generator finds some samples that can deceive the discriminator well, the generator will tend to generate these samples repeatedly without exploring other parts of the data distribution, resulting in a lack of diversity in the samples generated by the generator, which leads to mode collapse. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a time series prediction model training method based on data enhancement, solve the data scarcity problem in time series prediction, generate more diversified time series data, and improve the performance of the time series prediction model and the quality of generated samples.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] A time series prediction model training method based on data enhancement, comprising the steps of:

[0008] S1, divide the real time series data into the original training set and the test set;

[0009] S2. Building a MAD-TimeGAN network based on the TimeGAN network and using the MAD-GAN architecture, and using the MAD-TimeGAN network to generate sample time series data;

[0010] S3, merging the original training set with the sample time series data to obtain an enhanced training set;

[0011] S4, using the enhanced training set to train the time series prediction model;

[0012] S5. Use the test set to evaluate the prediction results of the time series prediction model.

[0013] The beneficial effects of the present invention are: providing a time series prediction model training method based on data enhancement, building a MAD-TimeGAN network based on the TimeGAN network and adopting the MAD-GAN architecture to generate more diverse sample time series data, and jointly forming a training set with real time series data to train the time series prediction model, so as to solve the data scarcity problem in time series prediction and improve the performance of the time series prediction model and the quality of generated samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is an overall flow chart of a time series prediction model training method based on data enhancement according to an embodiment of the present invention;

[0015] Figure 2 Diagram of the existing TimeGAN network training mechanism;

[0016] Figure 3 This is the overall framework diagram of the existing MAD-GAN;

[0017] Figure 4 Schematic diagram of different sequence generators being pushed into different modes;

[0018] Figure 5 This is the structural diagram of the improved MAD-TimeGAN of the present invention;

[0019] Figure 6 This is a visualization analysis result diagram of the Stock dataset in an embodiment of the present invention;

[0020] Figure 7 This is a visualization analysis result diagram of the Energy data set in an embodiment of the present invention;

[0021] Figure 8 This is a diagram showing the visual analysis results of the E1 data set in an embodiment of the present invention;

[0022] Fig. 9 This is a visualization analysis result diagram of the E2 data set in an embodiment of the present invention;

[0023] Fig.10 It is a line graph of the predicted value and actual value of the PSO-SVM model of the E1 data set in an embodiment of the present invention;

[0024] Fig.11 It is a line chart of the predicted value and actual value of the LSTM model of the E1 data set in an embodiment of the present invention;

[0025] Fig.12 It is a line graph of the predicted value and actual value of the PSO-SVM model of the E2 data set in an embodiment of the present invention;

[0026] Fig.13This is a line chart of the predicted value and actual value of the LSTM model of the E2 data set in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to explain the technical content, achieved objectives and effects of the present invention in detail, the following is an explanation in combination with the implementation modes and the accompanying drawings.

[0028] Please refer to Figures 1 to 13 , a time series prediction model training method based on data enhancement, comprising the steps of:

[0029] S1, divide the real time series data into the original training set and the test set;

[0030] S2. Building a MAD-TimeGAN network based on the TimeGAN network and using the MAD-GAN architecture, and using the MAD-TimeGAN network to generate sample time series data;

[0031] S3, merging the original training set with the sample time series data to obtain an enhanced training set;

[0032] S4, using the enhanced training set to train the time series prediction model;

[0033] S5. Use the test set to evaluate the prediction results of the time series prediction model.

[0034] From the above description, it can be seen that the beneficial effects of the present invention are: providing a time series prediction model training method based on data enhancement, building a MAD-TimeGAN network based on the TimeGAN network and adopting the MAD-GAN architecture to generate more diverse sample time series data, and jointly forming a training set with real time series data to train the time series prediction model, so as to solve the data scarcity problem in time series prediction and improve the performance of the time series prediction model and the quality of generated samples.

[0035] Furthermore, the MAD-TimeGAN network includes an automatic encoding component and an adversarial component;

[0036] The automatic encoding component includes an embedding function and a recovery function, and the adversarial component includes a sequence generator and a sequence discriminator;

[0037] The MAD-TimeGAN network includes at least two sequence generators.

[0038] Furthermore, in step S2, the MAD-TimeGAN network is used to generate sample time series data, specifically:

[0039] The MAD-TimeGAN network is used to train the auto-encoding component and the adversarial component, and the training conditions include auto-encoder parameter optimization, adversarial network parameter optimization, and learning of temporal features by the sequence generator.

[0040] Furthermore, the MAD-TimeGAN network is used to train the auto-encoding component and the adversarial component, specifically:

[0041] The embedding function and the recovery function are trained based on reconstruction loss and supervised loss, and the sequence generator and the sequence discriminator are trained based on unsupervised loss and supervised loss.

[0042] From the above description, we can know that the MAD-TimeGAN network is based on the TimeGAN network using the MAD-GAN architecture. The key to realizing TimeGAN is to jointly train the auto-encoding component and the adversarial component, so that TimeGAN can simultaneously learn encoding features, generate representations, and iterate across time; at the same time, the auto-encoding component and the adversarial component are jointly trained relying on three different loss functions. The embedding function of the auto-encoding component is committed to creating a latent space, while the adversarial component operates in the latent space. Finally, the temporal features of the synthetic data should be as similar as possible to the temporal features of the real data, that is, supervised learning. The potential dynamics of the real time series data and the generated sample time series data can be synchronized through supervised loss.

[0043] Furthermore, the step S2 also includes:

[0044] Determine the parameters of the MAD-TimeGAN network, including the time step, feature dimension, number of hidden layer units, number of iterations, and number of generators.

[0045] From the above description, it can be seen that by predetermining the various parameters of the MAD-TimeGAN network, it is ensured that the generated sample time series data is as consistent as possible with the real time series data.

[0046] Furthermore, the time series prediction model includes an LSTM model and a PSO-SVM model.

[0047] Furthermore, before step S4, the following steps are also included:

[0048] Parameters are set for the time series prediction model.

[0049] Furthermore, the parameter settings of the PSO-SVM model are specifically as follows:

[0050] Set the number of swarm particles to 10, the maximum number of iterations R = 100, the inertia weight w = 1, the learning factors c1 = 1.5, c2 = 1.7, and the ranges of the penalty coefficient c and the kernel function parameter g are both [0.1, 100];

[0051] The LSTM model parameter settings are specifically as follows:

[0052] Set the number of layers to 2, the number of iterations to 200, and the prediction result to be the average of 5 experiments.

[0053] From the above description, it can be seen that the time series prediction model can be implemented using different models, so as to subsequently verify the applicability of the data enhancement method of increasing the training set to different prediction models.

[0054] Furthermore, after step S2, the following steps are further included:

[0055] The real time series data and the sample time series data are subjected to t-distribution-stochastic nearest neighbor embedding t-SNE and principal component analysis PCA dimension reduction visualization processing.

[0056] From the above description, we can intuitively see how close the sample distribution of the sample time series data generated by the MAD-TimeGAN network is to the sample distribution of the real time series data, so as to preliminarily verify the effectiveness of the data augmentation method.

[0057] Furthermore, the step S5 also includes:

[0058] The mean absolute error (MAE), mean absolute percentage error (MAPE) and root mean square error (RMSE) are used as evaluation indicators of the prediction accuracy of the prediction results.

[0059] From the above description, we can see that the prediction accuracy is measured by three commonly used metrics in prediction, thereby effectively evaluating the effectiveness of data augmentation methods for small sample time series prediction.

[0060] The present invention provides a method for training a time series prediction model based on data enhancement, which is mainly used in a small sample time series prediction scenario, and is specifically described below in conjunction with a specific embodiment:

[0061] Please refer to Figure 1 , Embodiment 1 of the present invention is:

[0062] Time series prediction is a process of mapping historical time series with future time series targets. Assume that a window of T is given with n-dimensional historical time series x = {x1, x2, ···, x t ,x T},in Future time series target y = {xT+1 ,x T+2 ,···,x T+L},y∈R L , then the time series forecasting problem can be defined as:

[0063] f(x)+e=y (1);

[0064] Among them, f refers to the prediction function of the time prediction model, T refers to the historical input step, L refers to the future prediction step, and e represents the prediction error of the time prediction model.

[0065] Time series prediction is a data-driven prediction method. Its time series prediction model requires a large amount of training data to learn the feature dependencies of the data. When the training data is less, the accuracy of the prediction model will drop significantly. Therefore, this embodiment proposes a time series prediction model training method based on data enhancement. By enhancing the time series data, the training set data is expanded to solve the data shortage problem in time series prediction, so as to obtain a good prediction model.

[0066] Then a time series prediction model training method based on data enhancement in this embodiment is as follows: Figure 1 As shown, the steps include:

[0067] S1. Divide the real time series data into the original training set and the test set.

[0068] S2. Based on the TimeGAN network and using the MAD-GAN (Multi-Agent Diverse Generative Adversarial Networks) architecture, a MAD-TimeGAN network is constructed, and the MAD-TimeGAN network is used to generate sample time series data.

[0069] In this embodiment, the MAD-TimeGAN network consists of four parts, including an automatic encoding component and an adversarial component. The automatic encoding component includes an embedding function and a recovery function. The adversarial component includes a sequence generator and a sequence discriminator. At the same time, there are at least two sequence generators.

[0070] The key to implementing TimeGAN is to train the auto-encoder component and the adversarial component together, where the embedding function and the recovery function are trained based on the reconstruction loss and the supervised loss, and the sequence generator and the sequence discriminator are trained based on the unsupervised loss and the supervised loss. In this embodiment, the training conditions include the optimization of the auto-encoder parameters, the optimization of the adversarial network parameters, and the learning of temporal features by the sequence generator, so that TimeGAN can simultaneously learn encoding features, generate representations, and iterate across time.

[0071] That is, the auto-encoding component and the adversarial component are jointly trained by relying on three different loss functions. The embedding function of the auto-encoding component is committed to creating a latent space, while the adversarial component operates in the latent space. Finally, the temporal features of the synthetic data are made as similar as possible to the temporal features of the real data, that is, supervised learning. The potential dynamics of the real time series data and the generated sample time series data can be synchronized through supervised loss. In order to ensure that different generators capture different data distribution patterns, the discriminator must not only find real samples and fake samples, but also correctly identify the generator that produces a given fake sample.

[0072] S3. Merge the original training set with the sample time series data to obtain an enhanced training set.

[0073] S4. Use the enhanced training set to train the time series prediction model.

[0074] S5. Use the test set to evaluate the prediction results of the time series prediction model.

[0075] That is, in this embodiment, a MAD-TimeGAN network is constructed based on the TimeGAN network and the MAD-GAN architecture to generate more diverse sample time series data, which together with the real time series data constitute a training set to train the time series prediction model, so as to solve the data scarcity problem in time series prediction and improve the performance of the time series prediction model and the quality of generated samples.

[0076] In addition, in this embodiment, step S2 also includes:

[0077] Determine the parameters of the MAD-TimeGAN network, including the time step, feature dimension, number of hidden layer units, number of iterations, and number of generators.

[0078] That is, by predetermining the various parameters of the MAD-TimeGAN network, it is ensured that the generated sample time series data is as consistent as possible with the real time series data.

[0079] Embodiment 2 of the present invention is:

[0080] A time series prediction model training method based on data enhancement, based on the above embodiment 1, in order to verify the applicability of the data enhancement method to different time series prediction models, in this embodiment, the time series prediction model can adopt an LSTM model and a PSO-SVM model, so before step S4, it also includes:

[0081] Set the parameters of the LSTM model and the PSO-SVM model. The parameter settings of the PSO-SVM model can be:

[0082] Set the number of swarm particles to 10, the maximum number of iterations R = 100, the inertia weight w = 1, the learning factors c1 = 1.5, c2 = 1.7, and the penalty coefficient c and kernel function parameter g are both in the range of [0.1, 100].

[0083] The specific LSTM model parameter settings are:

[0084] Set the number of layers to 2, the number of iterations to 200, and the prediction result to be the average of 5 experiments.

[0085] That is, the time series prediction model can be implemented using different models, so that the applicability of the data augmentation method of increasing the training set to different prediction models can be verified later.

[0086] In addition, in this embodiment, after step S2, the following steps are further included:

[0087] The real time series data and sample time series data are visualized by t-distribution-random neighbor embedding t-SNE and principal component analysis PCA dimensionality reduction.

[0088] That is, we can intuitively see how close the sample distribution of the sample time series data generated by the MAD-TimeGAN network is to the sample distribution of the real time series data, so as to preliminarily verify the effectiveness of the data enhancement method.

[0089] Among them, t-SNE is a method that uses t distribution to measure the similarity between data samples in high-dimensional space. For each data sample in high-dimensional space, t-SNE calculates the similarity between it and other samples and converts the similarity into conditional probability distribution; in low-dimensional space, t-SNE also calculates the similarity between data samples and converts it into conditional probability distribution; then, t-SNE maps high-dimensional data to low-dimensional space by minimizing the Kullback-Leibler divergence between the conditional probability distributions between data samples in high-dimensional and low-dimensional space. In this embodiment, t-SNE is used for visual display through data dimensionality reduction to reflect the local similarity between generated data and original data, that is, the higher the similarity, the better the quality of generated data.

[0090] At the same time, principal component analysis PCA is a multivariate statistical analysis method that linearly transforms multiple variables to select a smaller number of important variables. The core idea is to obtain a new set of data dimensions by linearly transforming the dimensions of the original data. This new set of data dimensions is called principal components. The principal components are linearly independent. The principal components actually construct a new characteristic dimension of the data. Under this characteristic dimension, the dimension of the data may be compressed, and the calculation of the data will become easier, and it is also convenient for visualization. This embodiment uses principal component analysis to reduce the dimension of the data and visualize it, reflecting the global similarity between the generated data and the original data, that is, the higher the similarity, the better the quality of the generated data.

[0091] In addition, in this embodiment, in order to simulate the problem of insufficient data in actual situations, this embodiment performs data enhancement and prediction on two small sample data sets. After data enhancement on the training set data, the effectiveness of the data enhancement method for small sample time series prediction is evaluated by prediction error. In this embodiment, three commonly used metrics in prediction are used to measure the prediction accuracy, that is, step S5 also includes:

[0092] The mean absolute error (MAE), mean absolute percentage error (MAPE) and root mean square error (RMSE) are used as evaluation indicators of the prediction accuracy of the prediction results.

[0093] That is, the prediction accuracy is measured by three commonly used metrics in prediction, thereby effectively evaluating the effectiveness of data enhancement methods for small sample time series prediction.

[0094] Please refer to Figures 2 to 5 , Embodiment 3 of the present invention is:

[0095] A time series prediction model training method based on data enhancement. On the basis of the above-mentioned embodiment 2, during the TimeGAN training process, if the generator finds some samples that can deceive the discriminator well, the generator will tend to repeatedly generate these samples (i.e., sample time series data) without exploring other parts of the data distribution, resulting in a lack of diversity in the samples generated by the generator, which in turn leads to mode collapse. This embodiment improves the TimeGAN network by introducing MAD-GAN.

[0096] By introducing the MAD-GAN network, multiple generators are introduced to solve the limitations of data generation by a single generative model and generate more diverse time series data. MAD-GAN is a multi-agent GAN architecture consisting of k generators and 1 discriminator. In order to ensure that different generators capture different data distribution patterns, the discriminator must correctly identify the generator that produces a given fake sample in addition to finding real samples (i.e., real time series data) and fake samples (i.e., sample time series data).

[0097] In this embodiment, time series data generally consists of two elements: static features and temporal features. Static features refer to features that do not change over time; temporal features refer to features that change over time. Assume that S is a static feature, X is a temporal feature, S and X are random vectors belonging to an instance of S and X respectively, and the distribution of vectors S and X is: p(S, X 1:T ); The purpose of TimeGAN is to use real data to train a feature distribution Make it as close as possible to the original data feature distribution p(S,X 1:T ) are close, that is:

[0098]

[0099] where D is a measure of the distance between distributions.

[0100] TimeGAN consists of four parts: embedding function, recovery function, sequence generator and sequence discriminator. The embedding function and recovery function belong to the automatic encoding network, also called the automatic encoding component, and the sequence generator and sequence discriminator belong to the adversarial network, also called the adversarial component.

[0101] like Figure 2 The following figure shows the TimeGAN network training mechanism. Figure 2 The solid line in the middle represents the forward propagation of data, and the dotted line represents the backward propagation of gradients. e ,θ r ,θ g and θ d They represent the outputs of the embedding function, the recovery function, the sequence generator, and the sequence discriminator, respectively. The embedding function and the recovery function are trained based on the reconstruction loss and the supervision loss, and their objective functions are as follows:

[0102]

[0103] Among them, λ is a hyperparameter that balances the reconstruction loss and the supervision loss, and λ≥0, L S is the supervision loss, L Ris the reconstruction loss, and they are defined as:

[0104]

[0105] Among them, S and X 1:T are the static and temporal features of real time series data; X t represents the real time series data at time t, represents the sample time series data generated by the sequence generator at time t; 1:T represents the time series window length T, reflecting the time series change characteristics of the time series with a length of T, and p represents the characteristic distribution of the real time series data p(S,X 1:T );h S and h1: T They are S and X1 respectively: T The mapping vector in the latent space, and are the static and temporal features after data reconstruction, g X (h S ,h t-1 ,z t ) represents the output of the sequence generator, E S Indicates taking statistical average, h t represents the mapping from the temporal feature space of the real time series data at time t to the temporal feature space in the latent space, z t is a random vector,

[0106] The sequence generator and sequence discriminator are trained based on unsupervised loss and supervised loss, and their objective functions are as follows:

[0107]

[0108] Among them, η is a hyperparameter that balances the supervised loss and the unsupervised loss, and η ≥ 0. During the training process of the adversarial component, in addition to the unsupervised binary game on the classification accuracy, the sequence generator also minimizes the supervised loss and the unsupervised loss L U is defined as follows:

[0109]

[0110] Among them, y S With y t They represent the classification results of the static features and temporal features of the real time series data by the sequence discriminator, and The classification results of the sequence discriminator on the static features and temporal features of the sample time series data generated by the sequence generator.

[0111] From the above, we can see that the existing TimeGAN achieves data enhancement of time series by jointly training the auto-encoding network and the adversarial network. However, during the TimeGAN training process, if the sequence generator finds some samples that can deceive the sequence discriminator well, the sequence generator will tend to generate these samples repeatedly without exploring other parts of the data distribution, resulting in a lack of diversity in the samples generated by the sequence generator, which in turn leads to mode collapse.

[0112] Therefore, in order to solve the problems existing in the above-mentioned TimeGAN network and generate better quality and more diverse data, this embodiment improves the TimeGAN network by introducing MAD-GAN.

[0113] MAD-GAN is a multi-agent GAN architecture that contains at least two sequence generators and one sequence discriminator. In order to ensure that different sequence generators capture different data distribution patterns, the sequence discriminator must correctly identify the sequence generator that produces a given fake sample in addition to finding real samples and fake samples. The overall framework of MAD-GAN is as follows Figure 3 As shown in Figure 2, MAD-GAN consists of k sequence generators and 1 sequence discriminator. For the sequence generator objective function, in minimizing D k+1 (x;θ d ) values, the sequence discriminator is required to be able to identify the corresponding false samples The corresponding sequence generator. In the process of sequence generator training, the prior noise z~p z Input to each sequence generator, each sequence generator outputs the corresponding false sample For the i-th sequence generator, its model parameters Optimization is performed by minimizing the following objective function:

[0114]

[0115] Among them, D k+1 (x;θ d ) represents the probability that the (k+1)th observation sample is classified as true data, p d is the probability distribution of the real data, p z is the noise distribution.

[0116] The common goal of all sequence generators is to minimize the following objective function:

[0117]

[0118] Among them, p gi Represents the probability density function of the data generated by the i-th sequence generator.

[0119] For the sequence discriminator, its model parameter θ d The optimization is performed by maximizing the following objective function:

[0120] E x～p H(δ,D(x;θ d ))(10);

[0121] Among them, δ∈{0,1} k+1 , if the sample belongs to the jth sequence generator, then δ(j)=1,j∈{1,2,···,k}, otherwise δ(k+1)=1; H is the negative value of the cross entropy function.

[0122] Finally, in order to force the sequence generator to generate data with a wider distribution, this embodiment applies a minimum-maximum interaction condition between the sequence generator and the sequence discriminator, that is, minimizing the generator loss and maximizing the discriminator loss. The overall MAD-GAN objective function can be expressed as:

[0123]

[0124] From the above analysis, in order to correctly identify the sequence generator corresponding to the generated fake sample, the sequence discriminator must learn to push different sequence generators to different recognizable patterns, such as Figure 4 As shown, the objective of each generator is still the same as that of the generator in the standard GAN.

[0125] Figure 4 In the figure, M1 and M2 are pattern clusters, each cluster itself contains many patterns, and the arrows abstractly represent the gradients specific to the sequence generator.

[0126] By adopting the MAD-GAN architecture, the unsupervised loss L in MAD-TimeGAN U The definition is as follows:

[0127]

[0128] Among them, y s(k+1) and t(k+1) They represent the classification results of the static features and temporal features of the real data by the sequence discriminator, and They represent the classification results of the static features and temporal features of the generated data by the sequence discriminator, and y is the classification result of the real data and the generated data by the sequence discriminator.

[0129] Compared with the original TimeGAN, the improved method introduces multiple sequence generators, which can effectively alleviate the mode collapse problem, increase the diversity of generated samples, and improve the performance of the model and the quality of generated samples.

[0130] like Figure 5 The improved MAD-TimeGAN structure is shown, θ e ,θ r ,θ g i and θ d They represent the parameters of the embedding function, recovery function, sequence generator and sequence discriminator respectively, is the output of the sequence discriminator, and its result is the classification result y of the real data *(k+1) Or generate classification results for data

[0131] Among them, e is the embedding function and r is the recovery function, which together form the autoencoder. 1:T are the static and temporal features of the original data, h S and h 1:T As its mapping in latent space, and The static features and temporal features after data reconstruction. The static and temporal features are converted into their latent codes h through the embedding function e S ,h 1:T =e(S,X 1:T ), the latent code is restored to static and temporal features through the recovery function r The autoencoder implements a reversible mapping between feature space and latent space. Its function is to enable the adversarial network to learn static and temporal features in a space with smaller dimensions.

[0132] g is the generating function and d is the discriminant function, which together form an adversarial network. The generator generates a latent code through a tuple of static and temporal random vectors. The discriminator receives the static and temporal latent codes and returns them to the classifier: The symbol represents the real data h * Or generate data The same can be obtained The symbol represents the true data y of the classifier * and generate data

[0133] The original data and generated data are visualized by t-SNE and PCA dimensionality reduction. This method can intuitively show the closeness between the generated sample distribution and the original sample distribution, so as to preliminarily test the effectiveness of the data augmentation method.

[0134] In order to simulate the problem of insufficient data in actual situations, this embodiment performs data enhancement and prediction on two small sample data sets. After data enhancement of the training set data, the prediction error is used to evaluate the effectiveness of the data enhancement method for small sample time series prediction. This paper uses three commonly used metrics in prediction to measure prediction accuracy, including mean absolute error (MAE), mean absolute percentage error (MAPE) and root mean square error (RMSE). The formulas for the three metrics are as follows:

[0135]

[0136] where y i is the true value, Y i is the predicted value, and n is the number of samples.

[0137] Please refer to Figures 6 to 13 , Embodiment 4 of the present invention is:

[0138] A time series prediction model training method based on data enhancement, based on the above-mentioned embodiment 3, in this embodiment, actual data is provided for data enhancement to evaluate the prediction of the time series prediction model, as follows:

[0139] 1. Stock Dataset: This dataset covers Google’s historical stock data from 2004 to 2019, including 3624 sample points. Each sample includes six features: trading volume, highest price, lowest price, opening price, closing price, and adjusted closing price.

[0140] A benchmark experiment of data generation and visualization is conducted on this typical time series dataset. Due to the large number of samples in the dataset, visual analysis can be used to intuitively see the difference between the original data and the generated data. The original data and the generated data are visualized by t-distribution-stochastic nearest neighbor embedding (t-SNE) and principal component analysis (PCA) dimensionality reduction processing to intuitively see the degree of closeness between the generated sample distribution and the original sample distribution, so as to preliminarily test the effectiveness of the data augmentation method.

[0141] The parameters that need to be set for the TimeGAN and MAD-TimeGAN networks are as follows: time step, feature dimension, number of hidden layer units, number of iterations, etc. MAD-TimeGAN needs to set the number of generators. To determine the optimal MAD-TimeGAN model, experiments are conducted on the data augmentation network MAx-TimeGAN with x generators, where x is 2, 3, and 4. The specific parameter settings of TimeGAN and MAD-TimeGAN need to be determined according to the feature dimension and number of samples of the dataset.

[0142] Data enhancement experiments were performed on the Stock dataset using VAE, GAN, TimeGAN, and the MAD-TimeGAN model proposed in this embodiment. The original data and generated data were visualized and analyzed as shown in the figure. Figure 6 As shown, where x in MADx-TimeGAN represents the number of generators. Figure 6 The PCA graph in is used to observe the global similarity between the original data and the generated data, while the t-SNE graph is used to observe the local similarity between the original data and the generated data. The circular data points represent the original samples, and the triangular data points represent the generated samples. The VAE model only considers the characteristics of a single sample point, and does not consider the time dependency between the sample points, so its generated data effect is not ideal. The GAN model only performs simple generative adversarial training and also does not consider the correlation information of the time series, so the generated data distribution is quite different from the original data distribution. Compared with the first two generation models, the TimeGAN model considers the correlation between time series, and its generation effect is also improved, but it still has the common mode collapse problem in the GAN network. The reason is that during the training process of the TimeGAN model, if the generator finds some samples that can deceive the discriminator well, the generator will repeatedly generate these samples without exploring other parts of the data, which leads to a lack of diversity in the generated samples. For the MAD-TimeGAN model, its generated data can better cover the original data. The reason is that the MAD-TimeGAN model uses multiple generators for training at the same time during the training process, so that it can learn data of multiple different modes, thereby increasing the diversity of generated samples. MAD2-TimeGAN has the best data generation effect. It can also be observed from the figure that the MAD-TimeGAN model is not better with more generators. As the number of generators increases, the discriminator loss increases, and the quality of generated data also deteriorates. The data generation effect of the MAD4-TimeGAN model is not good.

[0143] The visualization analysis results show that the data augmentation of the Stock dataset has a higher coverage of the original data and is significantly better than the unimproved model.

[0144] 2. Energy Dataset: This dataset is a standard time series dataset from the UCI machine learning library, which records the relationship between various environmental variables and power consumption. There are 12,024 sample points in total, and each sample point includes 10 features.

[0145] The original data and generated data were visualized using t-SNE and PCA dimensionality reduction, with the same parameters as the above-mentioned Stock dataset.

[0146] Data enhancement experiments were performed on the Energy dataset using VAE, GAN, TimeGAN, and the MAD-TimeGAN model proposed in this embodiment. The original data and generated data were visualized and analyzed as shown in the figure. Figure 7 As shown, where x in MADx-TimeGAN represents the number of generators. Figure 7 The PCA graph in is used to observe the global similarity between the original data and the generated data, while the t-SNE graph is used to observe the local similarity between the original data and the generated data. The circular data points represent the original samples, and the triangular data points represent the generated samples. Similar to the results of the above Stock dataset, MAD2-TimeGAN has the best data generation effect, while MAD4-TimeGAN has a poor data generation effect. Therefore, in practical applications, the optimal MAD-TimeGAN model should be selected according to the actual situation.

[0147] The visualization analysis results show that the data augmentation of the Energy dataset has a higher coverage of the original data and is significantly better than the unimproved model.

[0148] 3. Monthly electricity data set (E1): This data set uses the monthly electricity data of a city in the south from January 2014 to December 2021, with a total of 96 sample points. Each sample point includes 7 features such as monthly total social electricity consumption, number of holidays, GDP, average temperature, perceived temperature, pressure, and visibility. The total social electricity consumption is the target prediction sequence.

[0149] First, the original data and generated data are visualized by t-distribution-stochastic nearest neighbor embedding (t-SNE) and principal component analysis (PCA) dimensionality reduction. Then, to simulate the problem of insufficient data in actual situations, this embodiment performs data enhancement and prediction on this small sample data set. After data enhancement of the training set data, the effectiveness of the data enhancement method for small sample time series prediction is evaluated by the prediction error, and PSO-SVM and LSTM are used to perform predictions respectively to verify the effectiveness of the data enhancement method for different prediction models.

[0150] The parameters that need to be set for the TimeGAN and MAD-TimeGAN networks are as follows: time step, feature dimension, number of hidden layer units, number of iterations, etc. MAD-TimeGAN requires additional setting of the number of generators. To determine the optimal MAD-TimeGAN model, experiments are conducted on the data augmentation network MAx-TimeGAN with x generators, where x is 2, 3, or 4 for analysis.

[0151] The PSO-SVM parameter settings are as follows: the number of swarm particles is 10, the maximum number of iterations R = 100, the inertia weight w = 1, the learning factors c1 = 1.5, c2 = 1.7, and the penalty coefficient c and kernel function parameter g are both in the range of [0.1, 100].

[0152] The LSTM parameter settings are as follows: the number of LSTM layers is set to 2, the number of iterations is set to 200, and the prediction results are the average of 5 experiments.

[0153] Data enhancement and prediction are performed on the small sample time series dataset E1, and the data generation visualization analysis is as follows Figure 8 As shown in the figure, for the E1 dataset, the generated data of the MAD2-TimeGAN model and the MAD4-TimeGAN model are closest to the original data distribution; on this basis, PSO-SVM and LSTM are used for prediction respectively to verify the effectiveness of the data enhancement method for different prediction models. The prediction results are shown in Table 1, Table 2, Fig.10 and Fig.11 shown.

[0154] Table 1: PSO-SVM prediction results

[0155]

[0156] Table 2: LSTM prediction results

[0157]

[0158]

[0159] As shown in Table 1, the prediction effect is very poor and the prediction accuracy is low when the original data is directly used for time series prediction. However, the prediction effect is significantly improved after data enhancement, and the data enhancement effect using the MAD-TimeGAN network is significantly better than TimeGAN. For the E1 dataset, since its dataset sample capacity is too small and the time dependence is weak, it is better to select PSO-SVM as the prediction model.

[0160] 4. Daily electricity consumption data set (E2): This data set records the daily electricity consumption data of a southern city from February 10, 2021 to December 26, 2022. There are 624 sample points in total. Each sample point includes four features: daily electricity consumption, average temperature, pressure, and dew point temperature. Daily electricity consumption is the target prediction sequence.

[0161] First, the original data and generated data are subjected to t-distribution-stochastic nearest neighbor embedding (t-SNE) and principal component analysis (PCA) dimensionality reduction visualization processing; then, in order to simulate the problem of insufficient data in actual situations, this embodiment performs data enhancement and prediction on this small sample data set. After data enhancement of the training set data, the effectiveness of the data enhancement method for small sample time series prediction is evaluated by the prediction error, and PSO-SVM and LSTM are used to perform predictions respectively to verify the effectiveness of the data enhancement method for different prediction models.

[0162] In the comparison, the parameters that need to be set for TimeGAN and MAD-TimeGAN networks, the PSO-SVM parameter settings, and the LSTM parameter settings are the same as those for the monthly electricity data set (E1).

[0163] Data enhancement and prediction are performed on the small sample time series dataset E2, and the data generation visualization analysis is as follows Fig. 9 As shown in the figure, for the E2 dataset, the MAD3-TimeGAN model has the best data generation effect. On this basis, PSO-SVM and LSTM are used for prediction respectively to verify the effectiveness of the data enhancement method for different prediction models. As shown in Tables 1 and 2, the prediction effect is very poor and the prediction accuracy is low when the original data is directly used for time series prediction. However, the prediction effect is significantly improved after data enhancement, and the data enhancement effect of using the MAD-TimeGAN network is significantly better than that of TimeGAN. For the E2 dataset, the dataset samples are relatively large and the time dependence is strong. Using LSTM as a prediction model is better.

[0164] In summary, whether using the LSTM model or the PSO-SVM model, the method of this embodiment has strong applicability and can improve the prediction accuracy to a certain extent. However, for different data sets, selecting a suitable prediction method can further improve the prediction accuracy. In actual application, the appropriate prediction method should be selected according to the characteristics of the data set. Fig.10 , Fig.11 , Fig.12 and Fig.13It can be seen that the time series prediction model training method based on data enhancement in this embodiment is more effective in predicting small sample time series, and its prediction curve is more consistent with the original curve. It can also make good predictions at some inflection points and peaks, indicating that the data synthesized by this method can better cover the original data.

[0165] Data enhancement experiments and visualization analysis were conducted on the Stock dataset and the Energy dataset. The visualization analysis results show that this method has a higher coverage of the original data and is significantly better than the unimproved model. At the same time, to further prove the effectiveness of this method for small sample time series prediction, data enhancement and prediction were performed on two small sample datasets. The prediction results show that this method is more effective in time series prediction, and for different prediction models, this method can have a good improvement effect.

[0166] In summary, the time series prediction model training method based on data enhancement provided by the present invention has the following beneficial effects: compared with the original TimeGAN, the improved method introduces multiple generators, which can effectively alleviate the mode collapse problem, increase the diversity of generated samples, and improve the performance of the model and the quality of generated samples; the data enhancement method provided by the present invention has higher coverage of the original data, and the effect is significantly better than the unimproved model; the data enhancement method provided by the present invention has a more significant effect on small sample time series prediction, and at the same time, it can have a good improvement effect on different prediction models.

[0167] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent transformations made using the contents of the present invention's specification and drawings, or directly or indirectly applied in related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A time series prediction model training method based on data enhancement, characterized in that: Includes steps: S1, divide the real time series data into the original training set and the test set; S2. Building a MAD-TimeGAN network based on the TimeGAN network and using the MAD-GAN architecture, and using the MAD-TimeGAN network to generate sample time series data; S3, merging the original training set with the sample time series data to obtain an enhanced training set; S4, using the enhanced training set to train the time series prediction model; S5. Use the test set to evaluate the prediction results of the time series prediction model.

2. The method for training a time series prediction model based on data enhancement according to claim 1, characterized in that: The MAD-TimeGAN network includes an automatic encoding component and an adversarial component; The automatic encoding component includes an embedding function and a recovery function, and the adversarial component includes a sequence generator and a sequence discriminator; The MAD-TimeGAN network includes at least two sequence generators.

3. The method for training a time series prediction model based on data enhancement according to claim 2, characterized in that: In step S2, the MAD-TimeGAN network is used to generate sample time series data, specifically: The MAD-TimeGAN network is used to train the auto-encoding component and the adversarial component, and the training conditions include auto-encoder parameter optimization, adversarial network parameter optimization, and learning of temporal features by the sequence generator.

4. The method for training a time series prediction model based on data enhancement according to claim 3, characterized in that: The adopting of the MAD-TimeGAN network to train the automatic encoding component and the adversarial component is specifically as follows: The embedding function and the recovery function are trained based on reconstruction loss and supervised loss, and the sequence generator and the sequence discriminator are trained based on unsupervised loss and supervised loss.

5. The method for training a time series prediction model based on data enhancement according to claim 1, characterized in that: The step S2 also includes: Determine the parameters of the MAD-TimeGAN network, including the time step, feature dimension, number of hidden layer units, number of iterations, and number of generators.

6. The method for training a time series prediction model based on data enhancement according to claim 1, characterized in that: The time series prediction model includes an LSTM model and a PSO-SVM model.

7. The method for training a time series prediction model based on data enhancement according to claim 6, characterized in that: Before step S4, the following steps are also included: Parameters are set for the time series prediction model.

8. The method for training a time series prediction model based on data enhancement according to claim 7, characterized in that: The parameter settings of the PSO-SVM model are specifically as follows: Set the number of swarm particles to 10, the maximum number of iterations R = 100, the inertia weight w = 1, the learning factors c1 = 1.5, c2 = 1.7, and the ranges of the penalty coefficient c and the kernel function parameter g are both [0.1, 100]; The LSTM model parameter settings are specifically as follows: Set the number of layers to 2, the number of iterations to 200, and the prediction result to be the average of 5 experiments.

9. The method for training a time series prediction model based on data enhancement according to claim 1, characterized in that: The step S2 further includes: The real time series data and the sample time series data are subjected to t-distribution-stochastic nearest neighbor embedding t-SNE and principal component analysis PCA dimension reduction visualization processing.

10. The method for training a time series prediction model based on data enhancement according to claim 1, characterized in that: The step S5 also includes: The mean absolute error (MAE), mean absolute percentage error (MAPE) and root mean square error (RMSE) are used as evaluation indicators of the prediction accuracy of the prediction results.

Images