A time series data enhancement method based on feature interaction time sequence generative adversarial network

Abstract

The application belongs to the technical field of data enhancement and deep learning, and relates to a time series data enhancement method based on a feature interaction time series generative adversarial network. The method integrates an explicit feature interaction mechanism and an association perception loss function in a conditional time series generative adversarial network framework, and the model is composed of four modules of an embedder, a restorer, a generator and a discriminator. First, the original multivariate time series data is preprocessed, time series latent variables containing feature correlation are extracted by combining the embedder, multi-head attention and a bidirectional long short-term memory network, and the restorer realizes reversible mapping between the latent space and the original data space; the generator generates pseudo latent variable sequences conforming to the feature correlation law in combination with multi-head attention, and the discriminator distinguishes true and false data from the global and local dimensions; and the model is jointly optimized and trained by using reconstruction loss, supervision loss, adversarial loss and correlation constraint loss. After training, the generator is used to generate a small number of high-fidelity time series samples to supplement the original data set and realize class balance.

Description

Technical Field

[0001] This invention belongs to the field of data augmentation and deep learning technology, and relates to a temporal data augmentation method based on feature-interaction temporal generative adversarial networks. Background Technology

[0002] Time-series data is widely used in numerous fields such as industrial process monitoring, financial transaction analysis, medical and health monitoring, and weather forecasting. In these applications, using deep learning models to classify or predict time-series data is a common technical approach. However, this type of method faces a prevalent and challenging problem in practical applications: the class distribution in the dataset is often extremely imbalanced. For example, in industrial equipment fault diagnosis, the number of samples in normal operating conditions far exceeds the number of samples in faulty conditions; in financial fraud detection, normal transaction data far outnumbers fraudulent transaction data. This class imbalance problem causes the trained classification model to be severely biased towards the majority class, resulting in extremely low performance in identifying or predicting the minority class, making it difficult to meet the high reliability requirements of practical applications. Furthermore, real-world time-series data is susceptible to sensor noise, communication packet loss, environmental interference, and other factors, leading to data gaps and noise pollution, further exacerbating the data quality shortcomings.

[0003] To alleviate the problem of data imbalance, various data augmentation methods have been proposed in existing technologies. Traditional data augmentation methods, such as synthetic minority oversampling and random oversampling, generate new samples by interpolation or simple duplication in the feature space. These methods are computationally simple but have significant drawbacks: First, they can only perform simple transformations within the local feature space of the original data, resulting in limited diversity of generated samples and difficulty in accurately reflecting the true distribution of complex time-series data; second, these methods disrupt the inherent inter-frame dependencies and physical constraints between features in time-series data, leading to unreasonable temporal logic in the generated samples.

[0004] In recent years, data augmentation methods based on deep learning, especially generative adversarial networks (GANs), have received widespread attention. For example, the TimeGAN model attempts to capture the dynamic patterns of time-series data by combining unsupervised adversarial training with supervised autoregressive learning. However, existing methods of this kind suffer from insufficient ability to capture the static and dynamic correlations between features. While models like TimeGAN can simulate the temporal trends of single features, they fail to fully model the inherent complex interactions between different features in multivariate time-series data. In many real-world scenarios, there are explicit physical or logical constraints between features of different dimensions. For example, in a certain process, temperature and pressure often show a positive correlation, and velocity and acceleration have a derivative relationship. Existing models cannot guarantee that the correlation matrix between features in the generated data is consistent with the real data, leading to generated samples that violate domain knowledge and lack rationality.

[0005] To address the aforementioned issues, there is an urgent need for a data augmentation method that can accurately capture the static and dynamic correlations between features in multivariate time series data and generate high-fidelity, highly diverse synthetic samples. This would effectively solve the problem of data class imbalance and provide balanced, high-quality training data support for various time series analysis tasks. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the present invention aims to provide a temporal data augmentation method based on Feature Interaction Temporal Generative Adversarial Network (FIT-GAN). By modeling the correlation between features in multivariate temporal data, high-quality synthetic samples that conform to the physical or logical laws of the original data are generated, fundamentally solving the model performance bias problem caused by class imbalance.

[0007] To achieve the above objectives, the technical solution of the present invention is as follows:

[0008] A temporal data augmentation method based on feature-interaction temporal generative adversarial networks (GANs) is disclosed. This method incorporates an explicit feature interaction mechanism and an association-aware loss function into the conditional temporal GAN ​​framework. Its model structure consists of four core modules: an embedder, a restorer, a generator, and a discriminator. The method includes the following steps:

[0009] Step 1: Obtain the original multivariate time series dataset. Each sample in the dataset contains multiple consecutive time frames, and each frame contains multiple dimensions of time series features and corresponding category labels.

[0010] Step 2, constructing a feature interaction temporal generative adversarial network, specifically including:

[0011] Step 2.1: Construct an embedder to map high-dimensional original time series data and static condition variables to a low-dimensional latent space. Before embedding the time series data, the embedder first calculates the correlation weights between each feature through a multi-head attention mechanism layer to generate correlation-weighted time series features. Then, it performs residual connection with the original input and inputs it into a bidirectional long short-term memory network to generate a sequence of time series latent variables that retains the static correlation between features.

[0012] Step 2.2: Construct a restorer to map the real latent variable sequence generated by the embedder or the pseudo latent variable sequence generated by the generator back to the original high-dimensional data space, thereby realizing the reversible mapping between the latent space and the original space.

[0013] Step 2.3: Construct a generator to generate a pseudo-latent variable sequence that is consistent with the distribution of the real latent variable sequence from a random noise distribution; the generator uses a long short-term memory network as the core time series generation unit and introduces a multi-head attention mechanism to calculate the correlation of the generated initial pseudo-latent variables to ensure that the feature association of the generated latent variables is consistent with the real data.

[0014] Step 2.4: Construct a discriminator to distinguish between real latent variable sequences and pseudo latent variable sequences from both global sequence and local time step dimensions, and drive the generator optimization through minimax game.

[0015] Step 3: Construct a joint loss function to perform multi-objective joint optimization training on the feature interaction temporal generative adversarial network; the joint loss function includes:

[0016] The reconstruction loss is used to constrain the embedder and the restorer, ensuring that the latent space can completely preserve the temporal information and feature correlation information of the original data.

[0017] The supervised loss is used to force the generator to learn the cross-time step dependencies of real time-series data;

[0018] The adversarial loss is used to achieve a minimum-maximum game between the generator and the discriminator, thereby pushing the generated data distribution to approximate the real data distribution.

[0019] Correlation constraint loss is used to explicitly align the feature correlation distributions of real data and generated data.

[0020] Step 4: Use the trained generator to generate target minority class time series samples that are consistent with the original data distribution and have reasonable feature associations, and supplement the generated samples into the original dataset to form a class-balanced augmented training dataset.

[0021] Preferably, the modeling process of the embedder in step 2.1 specifically includes:

[0022] Static conditional embedding: Obtaining static latent variables from category labels through non-linear mapping;

[0023] (1)

[0024] Temporal data embedding: The temporal feature sequence processed by the multi-head attention mechanism is input into the bidirectional long short-term memory network, and the temporal latent variables are generated step by step by combining the static latent variables with the previous latent state.

[0025] (2)

[0026] (3)

[0027] (4)

[0028] (5)

[0029] (6)

[0030] (7)

[0031] Preferably, the modeling process of the generator in step 2.3 specifically includes:

[0032] Static pseudo-latent variable generation: Static random noise is mapped nonlinearly to generate static pseudo-latent variables that are distributed in the same way as the real static latent variables;

[0033] (8)

[0034] Temporal pseudo-latent variable generation: The random noise sequence is input into the long short-term memory network, and the initial temporal pseudo-latent variables are generated step by step by combining the static pseudo-latent variables with the previous pseudo-latent state. Then, the correlation is calculated through a multi-head attention mechanism to obtain the final temporal pseudo-latent variable sequence.

[0035] (9)

[0036] (10)

[0037] Preferably, the formula for calculating the reconstruction loss in step 3 is:

[0038] (11)

[0039] (12)

[0040] in, and These represent the feature correlation matrices of the real data and the generated data, respectively. The KL divergence is used to constrain the overall consistency of the correlation distribution, and the mean squared error is used to constrain the element-wise similarity of the correlation values.

[0041] Preferably, the formula for calculating the supervision loss in step 3 is:

[0042] (13)

[0043] in This is the timing generation function of the generator, with static latent variables as input. Previous latent variables and random noise The output is the generated pseudo-time series latent variable at step t.

[0044] (14)

[0045] Preferably, the formula for calculating the adversarial loss in step 3 is:

[0046] (15)

[0047] Preferably, the formula for calculating the correlation constraint loss in step 3 is:

[0048] (16)

[0049] Among them, KL divergence is used to constrain the overall consistency of the correlation distribution, and mean squared error is used to constrain the element-wise similarity of the correlation values.

[0050] Compared with existing technical solutions, the beneficial effects of the present invention are as follows:

[0051] 1. Accurately capture the correlation between features: This invention introduces an explicit feature interaction layer in the embedder and generator, and designs a special correlation constraint loss function, so that the model can accurately learn and reproduce the static correlation and dynamic temporal dependence between features in real multivariate time series data. The generated synthetic samples are highly consistent with the original data in the feature correlation matrix and have sufficient physical or logical rationality.

[0052] 2. High quality of generated data: This invention ensures that the generated data closely matches the real time series data in terms of single feature time series trends, multi-feature association patterns, and global probability distribution by jointly optimizing reconstruction loss, supervision loss, adversarial loss, and correlation constraint loss.

[0053] 3. Improve the performance and robustness of downstream models: Training various mainstream time series classification or prediction models with the balanced dataset generated by this invention can stably and significantly improve the model's recognition accuracy, recall rate and F1 score, and enhance the model's stability and generalization ability under different training configurations, providing a high-quality data foundation for various highly reliable time series analysis tasks. Attached Figure Description

[0054] Figure 1 This is a diagram of the overall architecture of the model based on the feature-interaction temporal generative adversarial network of this invention;

[0055] Figure 2 This is an example diagram illustrating the working principle of the embedder and restorer of the present invention;

[0056] Figure 3 This is an example diagram illustrating the working principle of the generator and discriminator of this invention;

[0057] Figure 4 This is a schematic diagram of the collaboration between various modules and the optimization of the loss function in this invention;

[0058] Figure 5 This is a distribution map of each category of the original dataset in this embodiment of the invention;

[0059] Figure 6This is a visualization comparison of PCA dimensionality reduction of minority class samples generated by the present invention and existing techniques;

[0060] Figure 7 This is a heatmap comparison of the feature correlations of data generated by the present invention and existing technologies;

[0061] Figure 8 This is a comparison chart of the confusion matrix of the same classification model on the test set before and after enhancement using the method of this invention;

[0062] Figure 9 This is a graph showing the performance changes of the classification model under different synthetic sample injection volumes. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0064] A temporal data augmentation method based on Feature Interaction Temporal Generative Adversarial Network (FIT-GAN) includes the following steps:

[0065] Step 1, Data Acquisition and Preprocessing. The original dataset used in this embodiment is a multivariate time-series dataset. Each data sample contains one class label and sequence data of 12 consecutive time frames. Each frame contains features in 9 dimensions. For ease of explanation, it is assumed that the dataset has a total of 8136 samples, and the distribution of each class is as follows: Figure 5 As shown, the majority class samples are significantly more numerous than the minority class samples, indicating a serious imbalance problem. To eliminate the influence of different feature dimensions, the data for each feature dimension are normalized, scaling the values ​​to the [0,1] interval.

[0066] Step 2: Construct the FIT-GAN model.

[0067] Step 2.1, construct the embedder and restorer.

[0068] The design goal of embedders is to map high-dimensional raw time-series data to a low-dimensional latent space in order to extract core temporal dependencies and feature associations. Its modeling consists of two sub-processes: static conditional embedding and time-series data embedding.

[0069] Static conditional embedding: For a category label c, a one-hot encoder maps the label to static latent variables. .

[0070] Time series data embedding: For input time series data sequences ∈ (B is the batch size, T is the time step, and d is the feature dimension). First, the correlation weights between features are calculated through a multi-head attention mechanism layer. The specific calculation method is shown in formulas x to x. Subsequently, With the original input Residual connections are made to mitigate gradient vanishing and preserve original information. The results of the residual connections are then fed into a bidirectional long short-term memory (BiLSTM) network. The BiLSTM incorporates static latent variables. Compared to the previous potential state Generating time-series latent variables step by step Finally, the complete time-series latent variable sequence is output. . Figure 2 The working principle of the embedder and the restorer is illustrated by example.

[0071] The restorer is designed to achieve a reversible mapping from the latent space to the original data space. It employs a BiLSTM as the time-series recovery unit, targeting the real latent variable sequences generated by the embedder. Or a sequence of pseudo-latent variables generated by a generator. Mapping back the time-series data to the original dimensions step by step, and outputting the reconstructed sequence. .

[0072] Step 2.2: Construct the generator and discriminator.

[0073] The generator is designed to generate pseudo-latent variable sequences consistent with the true distribution from a random noise distribution in the latent space, thereby reducing the difficulty of generating them directly in the high-dimensional original space. Its modeling consists of two sub-processes:

[0074] Static pseudo-latent variable generation: static random noise that follows a standard normal distribution ∈ Static pseudo-latent variables with the same distribution as the true static latent variables are generated using the nonlinear mapping function MLP. ∈ .

[0075] Generating time-series pseudo-latent variables: generating random noise sequences ∈ Input LSTM, combined with static pseudo-latent variables Compared to the previous false underwater state Initial time-series pseudo-latent variables are generated step by step. Then, to ensure that the feature associations of the generated latent variables are consistent with the real data, the transpose of the feature correlation weight matrix α learned by the embeddinger is used to... Multi-head attention calculations are performed to obtain the final temporal pseudo-latent variables.

[0076] The discriminator is designed to evaluate the authenticity of generated data from both global sequence and local time step dimensions. It uses an LSTM as its core time-series discrimination unit, outputting an authenticity score for each time step for the input real or pseudo-time-series latent variable sequence. And the authenticity score of the entire sequence. Specifically, the discriminator internally contains a time-step-level discriminant function. and a sequence-level discriminant function . Given the current latent variables and historical sequences as input, output the probability of authenticity at this time step; Using the hidden state of the last time step of the LSTM as input, output the probability of the entire sequence being true. Figure 3 The working principle of the generator and discriminator is illustrated by example.

[0077] Step 3: Construct the joint loss function and train the model.

[0078] This embodiment employs a multi-loss joint optimization strategy, which balances the weights of each loss through hyperparameters to achieve collaborative training of the four modules. Figure 4 It demonstrates the collaboration and loss optimization process among the various modules.

[0079] Reconstruction loss: Used to constrain the embedder and restorer, ensuring that the latent space can completely preserve the core information of the original data. It consists of two parts: time-series data reconstruction loss and feature correlation reconstruction loss. The calculation formula is:

[0080] (17)

[0081] (18)

[0082] in, For the original input, For the rebuild output of the restorer; and These are the feature correlation matrices for the real data and the reconstructed data, respectively. This loss forces the latent space to simultaneously preserve time-step-by-time detailed information and the global correlation structure between features.

[0083] Supervised loss: Used to force the generator to learn the cross-time-step dependencies of real time-series data. This loss causes the generator to generate pseudo-latent variables under the constraints of "real static conditions + real historical time-series information," and calculates the mean squared error between these pseudo-latent variables and the real latent variables.

[0084] (19)

[0085] in, The true latent variables output by the embedder. This is the temporal generation function for the generator. This loss provides the generator with an accurate temporal dependency monitoring signal.

[0086] Adversarial loss: Used to implement a mini-max game between the generator and the discriminator. It covers both time-step and sequence-level discrimination, ensuring that the generated data is consistent with the real data in both local features and global distribution.

[0087] (20)

[0088] The first expectation term is derived from the real data distribution, and the second expectation term is derived from the generated data distribution. The optimization objective of the discriminator is to maximize... The optimization objective of the generator is to minimize .

[0089] Correlation-constrained loss: Used to explicitly align the feature correlation distributions of real and generated data. This loss combines KL divergence and mean squared error to constrain the similarity of the correlation matrix at both the overall distribution level and the element-wise numerical level.

[0090] (twenty one)

[0091] in, and These represent the feature correlation matrices of the real data and the generated data, respectively. KL divergence is used to constrain the overall consistency of the two correlation distributions, and mean squared error is used to constrain the numerical similarity of the correlation coefficients at corresponding locations.

[0092] Ultimately, the optimization goals for each module are:

[0093] Losses of Embedders and Restorers:

[0094] The generator's loss:

[0095] The discriminator's loss:

[0096] Wherein, λ1 and λ2 are hyperparameters that balance the importance of each loss term, and in this embodiment, they are set to 1 and 0.1, respectively. This embodiment uses mini-batch gradient descent to iteratively train the network until the model converges.

[0097] Step 4: Generate the augmented dataset and verify the effect.

[0098] The generator of the trained FIT-GAN model is used to generate a specified number of synthetic time-series samples of minority classes. Specifically, the generator parameters are fixed, and a random noise sequence is input. And the corresponding minority class label c, through forward propagation, generate a sequence of pseudo-latent variables. The data is then mapped back to the original data space via a restorer to obtain the final synthesized time-series sample. The generated synthetic samples are mixed with the majority and minority class samples from the original training data to form a class-balanced augmented training dataset.

[0099] To verify the effectiveness of this invention, multiple sets of comparative experiments can be designed. Accuracy, precision, recall, F1 score, JS divergence, and Frobenius norm are used as evaluation metrics. Various mainstream classification models (such as support vector machines, random forests, and deep belief networks) and time-series network models (such as TCN, LSTM, and GRU) are used as downstream validation models to compare the FIT-GAN of this invention with methods such as SMOTE, TabularGAN, and TimeGAN.

[0100] Typical data availability analysis results show that the classification model trained with FIT-GAN augmented data performs best on all validation models and evaluation metrics. Data similarity analysis results show that the JS divergence of FIT-GAN is significantly lower than that of the comparison methods across all categories. PCA dimensionality reduction visualization shows that the synthetic data generated by FIT-GAN achieves a high degree of distribution overlap with the original real data in the low-dimensional feature space. Feature correlation heatmap comparison shows that the feature correlation matrix of the FIT-GAN-generated data is highly consistent with the original data, and its Frobenius norm is much lower than that of TimeGAN, proving that this method can accurately restore the intrinsic correlation between features. Autoregressive prediction validation results show that the time series prediction model trained with FIT-GAN augmented data has a significantly reduced mean absolute error, proving that the generated data has a reasonable temporal evolution pattern.

[0101] Comparative experiments with different synthetic sample injection amounts show that as the amount of synthetic samples generated by FIT-GAN increases, the average accuracy of the classification model increases stepwise, reaching a performance peak at an optimal injection amount. Further increasing the sample amount leads to a slight performance decrease, thus determining the optimal synthetic sample injection amount.

[0102] In summary, the time series data augmentation method based on feature-interactive temporal generative adversarial networks proposed in this invention can effectively generate high-quality time series samples that conform to the inherent physical or logical laws of the original data, accurately fill the distribution gaps of minority class samples, provide balanced and high-quality training data support for various time series analysis models, and improve the recognition performance and robustness of the models on imbalanced datasets from the source.

[0103] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A temporal data augmentation method based on feature-interactive temporal generative adversarial networks, characterized in that, Includes the following steps: Step 1: Obtain the original multivariate time series dataset. Each sample in the dataset contains multiple consecutive time frames, and each frame contains multiple dimensions of time series features and corresponding category labels. Step 2, constructing a feature interaction temporal generative adversarial network, specifically including: Step 2.1: Construct an embedder to map high-dimensional original time series data and static condition variables to a low-dimensional latent space. Before embedding the time series data, the embedder first calculates the correlation weights between each feature through a multi-head attention mechanism layer to generate correlation-weighted time series features. Then, it performs residual connection with the original input and inputs it into a bidirectional long short-term memory network to generate a sequence of time series latent variables that retains the static correlation between features. Step 2.2: Construct a restorer to map the real latent variable sequence generated by the embedder or the pseudo latent variable sequence generated by the generator back to the original high-dimensional data space, thereby achieving a reversible mapping between the latent space and the original space. Step 2.3: Construct a generator to generate a pseudo-latent variable sequence that is consistent with the distribution of the real latent variable sequence from the random noise distribution. The generator uses a long short-term memory network as the core time series generation unit and introduces a multi-head attention mechanism to calculate the correlation of the generated initial pseudo-latent variables to ensure that the feature association of the generated latent variables is consistent with the real data. Step 2.4: Construct a discriminator to distinguish between real latent variable sequences and pseudo latent variable sequences from both global sequence and local time step dimensions, and drive the generator optimization through minimax game. Step 3: Construct a joint loss function to perform multi-objective joint optimization training on the feature interaction temporal generative adversarial network; The joint loss function includes: Reconstruction loss is used to constrain the embedder and the restorer to ensure that the latent space can completely preserve the temporal information and feature correlation information of the original data; supervision loss is used to force the generator to learn the cross-time step dependencies of the real temporal data; adversarial loss is used to realize the minimax game between the generator and the discriminator to push the distribution of generated data to approximate the distribution of real data; correlation constraint loss is used to explicitly align the feature correlation distributions of real data and generated data. Step 4: Use the trained generator to generate target minority class time series samples that are consistent with the original data distribution and have reasonable feature associations, and supplement the generated samples into the original dataset to form a class-balanced augmented training dataset.

2. The temporal data augmentation method based on feature-interaction temporal generative adversarial networks as described in claim 1, characterized in that, In step 2.1, the modeling process of the embedder specifically includes: Static conditional embedding: Obtaining static latent variables from category labels through non-linear mapping; (1) Temporal data embedding: The temporal feature sequence processed by the multi-head attention mechanism is input into a bidirectional long short-term memory network, and temporal latent variables are generated step by step by combining the static latent variables with the previous latent state; (2) (3) (4) (5) (6) (7)。 3. The temporal data augmentation method based on feature-interaction temporal generative adversarial networks as described in claim 1, characterized in that, In step 2.3, the modeling process of the generator specifically includes: Static pseudo-latent variable generation: Static random noise is mapped nonlinearly to generate static pseudo-latent variables that are distributed in the same way as the real static latent variables; (8) Temporal pseudo-latent variable generation: The random noise sequence is input into the long short-term memory network, and the initial temporal pseudo-latent variables are generated step by step by combining the static pseudo-latent variables with the previous pseudo-latent state. Then, the correlation is calculated through a multi-head attention mechanism to obtain the final temporal pseudo-latent variable sequence. (9) (10)。 4. The temporal data augmentation method based on feature-interaction temporal generative adversarial networks as described in claim 1, characterized in that, In step 3, the formula for calculating the reconstruction loss is: (11) (12) in, and These represent the feature correlation matrices of the real data and the generated data, respectively. The KL divergence is used to constrain the overall consistency of the correlation distribution, and the mean squared error is used to constrain the element-wise similarity of the correlation values.

5. The temporal data augmentation method based on feature-interaction temporal generative adversarial networks as described in claim 1, characterized in that, In step 3, the formula for calculating the supervision loss is: (13) in This is the timing generation function of the generator, with static latent variables as input. Previous latent variables and random noise The output is the generated pseudo-time series latent variable at step t; (14)。 6. The temporal data augmentation method based on feature-interaction temporal generative adversarial networks as described in claim 1, characterized in that, In step 3, the formula for calculating the adversarial loss is: (15) Preferably, the formula for calculating the correlation constraint loss in step 3 is: (16) Among them, KL divergence is used to constrain the overall consistency of the correlation distribution, and mean squared error is used to constrain the element-wise similarity of the correlation values.

Images