Generative adversarial network method for electronic nose multivariable time series data enhancement

Abstract

The invention discloses a generative adversarial network method for electronic nose multivariable time series data enhancement, which comprises the following steps: independently fitting a Gaussian mixture model for each category of data in an electronic nose data set, and carrying out adversarial training on a generative adversarial network which comprises a generator and a discriminator; enhanced data is generated using the trained generator. According to the method, the distribution consistency constraint is applied between the noise potential space and the generated data, the common mode collapse problem of the generative adversarial network in the time sequence data generation is relieved, the generated data better conforms to the real data distribution, the adopted total loss LR3 is combined with the gradient penalty strategy, and the time sequence data generation efficiency is improved. The stability of model training is remarkably improved, the common instability problem in the generative adversarial network training process is relieved, and the quality of generated data is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of sensor data processing, and in particular to a method for enhancing lung cancer electronic nose data. Background Art

[0002] The diagnosis of lung cancer is crucial to improving patient survival rates, and electronic nose-based lung cancer detection provides a new, non-invasive, rapid, and economical approach for detecting lung cancer markers. However, similar to other medical datasets, existing lung cancer time series datasets suffer from limited sample size and class imbalance, which limits the generalization performance of advanced deep learning techniques on these datasets. To address this challenge, generative models such as Generative Adversarial Networks (GANs) have been widely used in time series data generation tasks due to their powerful data generation capabilities. By using data augmentation synthesized by generative models, the performance of deep learning models on small datasets can be effectively improved.

[0003] Traditional generative adversarial networks generally use standard Gaussian noise or uniformly distributed noise as the input noise of the generator G, that is: Or z~U(-1,1). Here, z represents the input noise. However, noise that follows a single distribution has a relatively limited ability to fit multimodal data. Mode collapse occurs when the generator's ability to generate data during training is limited to a few modes or categories, failing to cover the full distribution of real data.

[0004] Because traditional GAN models face not only mode collapse but also training instability when generating multi-modal time series, there is an urgent need to address these technical challenges to improve the quality of generated data.

[0005] Lung cancer-related data collected by electronic nose systems is typically expressed as a multivariate time series (MTS), consisting of signals from multiple sensor channels. MTS data is composed of multiple different univariate time series (UTS). Compared to univariate data, MTS data has richer feature information, but also introduces more complex inter-channel correlations. During the generation process, in addition to ensuring the quality of each variable within the data, it is also necessary to ensure the relationships between each variable. Therefore, how to generate high-quality lung cancer MTS data is a technical challenge that needs to be solved. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a generative adversarial network method for electronic nose multivariate time series data enhancement, so as to solve the technical problem of using a generative adversarial network to generate lung cancer MTS data, and to solve the technical problem of avoiding network model collapse and training instability in the process of generating lung cancer MTS data.

[0007] The generative adversarial network method for enhancing multivariate time series data of an electronic nose according to the present invention comprises the following steps:

[0008] I) For the multivariate time series data collected by the gas sensor array of the electronic nose, one gas sensor corresponds to one data channel. The linear discriminant analysis method is used to remove redundant channel data. The linear discriminant function is as follows:

[0009] g(F)=W T F (1)

[0010] Where W=[w1,w2,…,w j ,…,w d ] T is a weight vector whose elements are calculated from the intra-class scatter matrix and the inter-class scatter matrix; F = f1, f2, ..., f j ,…,f d ] T is the eigenvector of the gas sensor array, j∈{1,2,…,d} represents the channel number, d is the number of sensors, f j represents the characteristic value of the jth gas sensor, f j Calculated from the maximum signal value of the jth channel:

[0011]

[0012] Among them, the function g(X k ,j)Return sample X k The j-th dimension time series S j , max(·) returns S j The maximum value of all elements in ;

[0013] II) After removing redundant channel data, a Gaussian mixture model is fitted for the data from different channels in the same category of data. Each Gaussian mixture model is expressed as:

[0014]

[0015] Among them, V i,j is the Gaussian mixture model corresponding to the jth channel of the i-th category, i∈{1,2,…,C} represents the category number, k i,j Indicates V i,j kth i,jGaussian components, K i,j Represents the total number of Gaussian components corresponding to the j-th channel of category i, π i,j,k The kth channel corresponding to the jth channel of category i i,j The weights of the Gaussian components, The representative mean is The covariance is Gaussian distribution;

[0016] Each sample in a multivariate time series dataset is assigned an index pair (i, j, k i ), used for noise generation of different patterns within a class;

[0017] III) performing adversarial training on a generative adversarial network, which includes a generator and a discriminator. The adversarial training process is divided into two stages, and each stage of adversarial training includes:

[0018] 1) Gaussian mixture model is based on the label and index of the sample (i, j, k i ) Find the corresponding Gaussian component k i,j , for Gaussian component k i,j Sampling Gaussian mixed noise,

[0019]

[0020] in, is Gaussian mixture noise;

[0021] For each sample of category i, its noise latent space vector is expressed as:

[0022]

[0023] where k i =[k i,1 ,k i,2 ,…,k i, d] represents the combination of Gaussian component numbers of the Gaussian mixture model corresponding to different channels in the sample of category i;

[0024] 2) Mix the Gaussian noise As the input noise of the generator, the generator generates fake samples;

[0025] 3) The fake samples generated by the generator and the real samples from the real data are input into the discriminator, which judges the fake samples and the real samples to obtain the discrimination results;

[0026] The generator and the discriminator both include several group transformation modules containing channel adaptive graph attention, and the group transformation modules containing channel adaptive graph attention include a channel group transformation module that performs linear transformation on each pass data, a channel adaptive graph attention module connected to the output of the channel group transformation module, and a LeakyRelu activation function layer connected to the output of the channel adaptive graph attention module;

[0027] The channel grouping transformation module splits the input multivariate time series data into univariate time series data, and then inputs each univariate time series data into different linear layers for linear transformation, and then splices the output data of each linear layer. The spliced multivariate time series data features are then input into the channel adaptive graph attention module;

[0028] The channel adaptive graph attention module processes the data output by the grouped linear transformation module as follows:

[0029] ①Calculate channel characteristics:

[0030]

[0031] Among them, X :,t,: Represents the data of all batches and all channels of the input channel adaptive graph attention module at the t-th time step, and L represents the length of the input data;

[0032] ②Build a dynamic adjacency matrix between channels:

[0033] Perform linear transformation on the channel features and generate a normalized adjacency matrix A as the channel interaction weight:

[0034]

[0035] in, is the weight matrix, b is the bias vector, and the matrix is normalized using Softmax;

[0036] ③ Information fusion of channel association relationships:

[0037] The dynamic adjacency matrix A is used to weight the data X of the input channel adaptive graph attention module:

[0038] X′=X·A (8)

[0039] in, Where B represents the batch size of the data, L represents the length of the data, and J represents the number of channels of the data; The weighted result obtained by multiplying the two Same shape as X;

[0040] ④Residual feature fusion:

[0041] X out =X′+X (9)

[0042] Fusion DataX out Enter the LeakyRelu activation function layer;

[0043] In the first stage of the adversarial training process, the total loss L is introduced latent Acting on the generator and introducing the total loss L R3 Acts on both the generator and the discriminator; in the second stage of the anti-training training process, only the total loss L is introduced R3 Acts on both the generator and the discriminator;

[0044] Total loss L latent The expression is as follows:

[0045] L latent =L cst +λL spec (10)

[0046] Among them, L cst is the consistency loss, L spec is spectrum loss; is a weight hyperparameter used to balance the contribution of consistency and spectral loss;

[0047] Consistency loss L cst Used to constrain the distribution between the data generated by the generator and the real data, L cst The expression is as follows:

[0048]

[0049] in, For the generator Fake samples generated for input; is the real data, which belongs to category i and contains all channels;

[0050]

[0051] in, is the category i, channel j and belongs to the kth i,j True samples of clusters, j∈{1,2,…,d};

[0052] Spectrum loss L spec It is used to constrain the spectrum similarity between the data generated by the generator and the real data, L spec The expression is as follows:

[0053]

[0054] Where STFT(·) represents the short-time Fourier transform operation;

[0055] During the training process, the total loss L cst Backward pass to the generator to constrain the distribution consistency between the latent space and the generated sample space;

[0056] Total loss L R3 The expression is as follows:

[0057]

[0058] L RpGAN The loss is described as follows:

[0059]

[0060] Among them, p z (z) is the distribution of noise z, p data (x) is the distribution of the real sample x, D(G(z)) is the discriminant output of the discriminator for the generated sample G(z), D(x) is the discriminant output of the discriminator for the real sample x, and function f is the Softplus function;

[0061] Gradient penalty term L R1 The description is as follows:

[0062]

[0063] Where L is the length of the input time series; d is the number of sensors, that is, the total number of variable dimensions of the time series; is the partial derivative of the discriminator with respect to the value of the j-th variable at the l-th time step, that is, the gradient of the data point;

[0064] Gradient penalty term L R2 The description is as follows:

[0065]

[0066] in, Represents the partial derivative of the discriminator with respect to the value of the j-th variable at the l-th time step in the generated data, that is, the gradient of the data point;

[0067] γ is a hyperparameter used to control the gradient penalty term L R1 and L R2 Strength;

[0068] During the training process, the total loss L R3 Backward pass to the generator and discriminator;

[0069] IV) The Gaussian mixture model finds the corresponding Gaussian components according to the labels and indices of the samples in the electronic nose dataset, samples the Gaussian components to obtain Gaussian mixture noise, and inputs the Gaussian mixture noise into the generative adversarial network trained in step III), and the generator generates enhanced data.

[0070] Furthermore, the network structure of the generator in step III) includes four sequentially connected group transformation modules containing channel adaptive graph attention and a Tanh activation function layer connected to the output of the fourth group transformation module containing channel adaptive graph attention; the network structure of the discriminator in step III) includes three group transformation modules containing channel adaptive graph attention, a first convolutional layer connected to the output of the third group transformation module containing channel adaptive graph attention, a first LeakyRelu activation function layer connected to the output of the first convolutional layer, a second convolutional layer connected to the output of the first LeakyRelu activation function layer, a second LeakyRelu activation function layer connected to the output of the second convolutional layer, and a Sigmoid function layer connected to the output of the second LeakyRelu activation function layer.

[0071] Beneficial effects of the present invention:

[0072] 1. The generative adversarial network method for electronic nose multivariate time series data enhancement in the present invention alleviates the common mode collapse problem of generative adversarial networks in time series data generation by imposing distribution consistency constraints between the noise latent space and the generated data, making the generated data more consistent with the real data distribution.

[0073] 2. The present invention is used for the generative adversarial network method for electronic nose multivariate time series data enhancement, using a total loss L R3 Combined with the gradient penalty strategy, the stability of model training is significantly improved, the instability problem common in the training process of generative adversarial networks is alleviated, and the quality of generated data is improved.

[0074] 3. A notable characteristic of the electronic nose's gas sensor array is cross-sensitivity, that is, a single sensor may respond to multiple gases, resulting in a high degree of redundancy between channels. The data collected by some redundant channels contributes less to the multivariate time series (MTS) classification task and may even reduce the overall accuracy of the classification model. In addition, the existence of redundant channels increases the data dimension, thereby increasing the complexity of model calculation and modeling. The present invention removes redundant channel data through linear discriminant analysis and screens out sensor channels that contribute more to the classification task. After optimizing the sensor array in this way, it can not only improve the classification performance of downstream tasks, but also reduce computational overhead.

[0075] 4. The MTS is composed of multiple different UTSs. During the generation process, in addition to ensuring the quality of each variable, the relationship between each variable must also be maintained. Because the MTS curve of the lung cancer electronic nose is primarily related to the timing and concentration of the gas passing through the sensor array when collecting sensor array data, different sensors may have similar response trends. This is the main source of inter-channel correlation in lung cancer MTS data. However, given that the sensors themselves have different sensitivities to different gases and the sensor response processes are relatively independent, if the model overemphasizes this correlation, it may lead to a decrease in the independence of the data from different channels of the generated data. Therefore, the model should not blindly emphasize the relationship between channels.

[0076] In view of the relatively strong independence between different channels in the lung cancer MTS dataset, the present invention constructs a noise latent space vector Decoupling the latent space through channels not only enhances the model's ability to perceive data from different channels, but also makes the data generated by the model more consistent with real-world sensor array designs. This not only helps the model-generated data better align with real-world patterns but also facilitates further application and analysis of the generated data (for example, generating data on theoretically plausible trends).

[0077] 5. The present invention performs channel decoupling within the latent space, that is, within the noise latent space received by the generator, the data of different channels come from different distributions. This means that the generator is already able to obtain a channel-separated initialization input, which helps the generator quickly distinguish time series data from different channels during the training phase. However, if the processing process within the generator is not adjusted accordingly, and the network architecture for processing UTS is still used for direct transformation, it is likely that the initialization of channel separation will not play the expected role - because the generator designed for UTS itself will tend to use a single, shared transformation process to process the input multi-channel noise. Since the time series between different channels may have large differences in patterns, some patterns may be more complex and some patterns may be simpler, and their response process is closely related to the physical design of the sensor itself. If the generative model uses a unified parameter weight for learning throughout the entire processing process, the model will tend to process the data of all channels in a unified manner, resulting in excessive mixing of information between different channels, making it impossible to achieve true channel decoupling.

[0078] If MTS data is directly expanded in one dimension, the number of model parameters will increase dramatically as the length and dimension of the time series increase. Furthermore, if data from different channels in the MTS are concatenated end-to-end, the data near the beginning and end of the generated time series will behave abnormally. This behavior is influenced by the other connected channel sequences and does not conform to the response patterns of real-world sensors, significantly impacting the quality of the generated data.

[0079] To solve the above problems, the present invention uses a channel grouping transformation module in the production adversarial network. The channel grouping transformation module groups the MTS data according to the number of channels, and each channel is a group, and the number of groups is the number of channels. Independent parameters and weights are used within each group, and only data from one channel is processed. Since the parameters and weights between groups are independent of each other, the model can automatically adapt to the patterns and transformation methods of different channel data during the training process, thereby establishing a relatively separated learning goal between different channels and achieving channel decoupling during the training process. In this way, the model can enhance the relative independence of different channel data during the generation process.

[0080] 6. Although completely independent channel data helps the model learn the independent features of each channel data and gives the model channel-independent generation capabilities, its neglect of the correlation between channel data changes will, to a certain extent, lead to less realistic data generation. Even if the sensors corresponding to each channel are different, the types of gases they can detect are different, and the patterns of the response curves are also different, the time it takes for the gas to reach the sensor and the time it takes for the sensor to start and end responding are roughly the same; in addition, there may be related response trends between different channels of each sample. Therefore, the present invention proposes a channel-adaptive graph-inspired attention module (CAGAmodule) to model the correlation between channels, and sets learnable channel interaction weights for dynamic learning to enhance the model's attention to different channels, thereby further improving the quality of generated data.

[0081] 7. This invention employs a two-stage training mechanism during model training: the first stage introduces latent space distribution consistency constraints and R3 regularization to strengthen the generator's initial alignment with the real data distribution, improving the model's recognition and coverage of latent patterns; the second stage removes the distribution consistency constraints, retaining only R3 regularization, further stabilizing the model training process and improving the diversity and authenticity of generated samples. This two-stage training mechanism enables a gradual transition from generator learning of distribution structure to stable optimization of generation quality, improving overall training efficiency and model robustness, and preventing the negative impact of early pattern collapse on subsequent training. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] Figure 1 This is the MTS-GMN-GAN framework diagram.

[0083] Figure 2 This is the structural diagram of the channel grouping transformation module.

[0084] Figure 3Figure 2 is the structural diagram of the channel-adaptive graph attention module.

[0085] Figure 4 Detailed structure of the grouped transformation module with channel-adaptive graph attention.

[0086] Figure 5 This is the generator network structure diagram.

[0087] Figure 6 This is the discriminator network structure diagram.

[0088] Figure 7 Validation accuracy on the lung cancer dataset validation set when the number of sensors changes.

[0089] Figure 8 Direct visual comparison of MTS data synthesized for different models.

[0090] Figure 9 Direct visual comparison of MTS data synthesized for different models.

[0091] Figure 10 Visual comparison of MTS data synthesized by different models after PCA dimensionality reduction.

[0092] Figure 12 The effect of using different generative models for data augmentation on the classification performance of each classification model.

[0093] Figure 13 The classification performance of different classification models on the test set changes with the increase in the number of generated data. DETAILED DESCRIPTION

[0094] The present invention will be further described below with reference to the accompanying drawings and examples.

[0095] The generative adversarial network method for enhancing multivariate time series data of an electronic nose in this embodiment includes the following steps:

[0096] I) For the multivariate time series data collected by the gas sensor array of the electronic nose, one gas sensor corresponds to one data channel. The linear discriminant analysis method is used to remove redundant channel data. The linear discriminant function is as follows:

[0097] g(F)=W T F (1)

[0098] Where W=[w1,w2,…,w j ,…,w d ] T is a weight vector whose elements are calculated from the intra-class scatter matrix and the inter-class scatter matrix; F = f1, f2, ..., f j ,…,f d ]T is the eigenvector of the gas sensor array, j∈{1,2,…,d} represents the channel number, d is the number of sensors, f j represents the characteristic value of the jth gas sensor, f j Calculated from the maximum signal value of the jth channel:

[0099]

[0100] Among them, the function g(X k ,j)Return sample X k The j-th dimension time series S j , max(·) returns S j The maximum value of all elements in .

[0101] II) After removing redundant channel data, a Gaussian mixture model (GMM) is fitted for the data from different channels in the same category. Each Gaussian mixture model is expressed as:

[0102]

[0103] Among them, V i,j is the Gaussian mixture model corresponding to the jth channel of the i-th category, i∈{1,2,…,C} represents the category number, k i,j Indicates V i,j kth i,j Gaussian components, K i,j Represents the total number of Gaussian components corresponding to the jth channel of category i, π i,j,k The kth channel corresponding to the jth channel of category i i,j The weights of the Gaussian components, The representative mean is The covariance is Gaussian distribution.

[0104] Each sample in a multivariate time series dataset is assigned an index pair (i, j, k i ), used for noise generation of different patterns within a class.

[0105] III) Conduct adversarial training on a Generative Adversarial Network (GAN), which includes a generator and a discriminator. The adversarial training process is divided into two stages. Each stage of adversarial training includes:

[0106] 1) Gaussian mixture model is based on the label and index of the sample (i, j, k i ) Find the corresponding Gaussian component k i,j, for Gaussian component k i,j Sampling to obtain Gaussian Mixture Noise (GMN),

[0107]

[0108] in, is a Gaussian mixture noise.

[0109] Compared to traditional single-distribution noise, the GMN obtained by this invention can effectively increase the diversity of generated data during training, making it more closely match the distribution characteristics of real data, thereby significantly alleviating the problem of mode collapse. In addition, because the GMM is fitted based on real data, its statistical characteristics are closer to the actual distribution of the target data, which further improves the quality of the synthesized data.

[0110] 2) Mix the Gaussian noise As the input noise of the generator, the generator generates fake samples.

[0111] In addition, the model structure adaptive logic can be introduced to dynamically adjust the number of layers of the generator and discriminator using a logarithmic scale according to the length of the time series. This mechanism can automatically adapt to univariate time series data of different lengths without manually modifying the model structure, making the GMN-GAN model in this invention more scalable and able to adapt to data generation tasks of time series of different lengths.

[0112] 3) The fake samples generated by the generator and the true samples from the real data are input into the discriminator, and the discriminator judges the authenticity of the fake samples and the true samples to obtain the discrimination results.

[0113] In the present invention, the generator and the discriminator both include several group transformation modules containing channel adaptive graph attention, and the group transformation modules containing channel adaptive graph attention include a channel group transformation module that performs linear transformation on each passing data, a channel adaptive graph attention module connected to the output of the channel group transformation module, and a LeakyRelu activation function layer connected to the output of the channel adaptive graph attention module. The structure of the channel group transformation module is as follows: Figure 2 As shown in the figure, the structure of the channel adaptive graph attention module is as follows Figure 3 As shown in the figure, the structure of the group transformation module with channel adaptive graph attention is as follows Figure 4 In the present invention, the network structure of the generator is as follows Figure 5 As shown, it includes four sequentially connected group transformation modules with channel adaptive graph attention and a Tanh activation function layer connected to the output of the fourth group transformation module with channel adaptive graph attention. In the present invention, the network structure of the discriminator is as follows Figure 6As shown, it includes three group transformation modules with channel adaptive graph attention, a first convolutional layer connected to the output of the third group transformation module with channel adaptive graph attention, a first LeakyRelu activation function layer connected to the output of the first convolutional layer, a second convolutional layer connected to the output of the first LeakyRelu activation function layer, a second LeakyRelu activation function layer connected to the output of the second convolutional layer, and a Sigmoid function layer connected to the output of the second LeakyRelu activation function layer.

[0114] The channel grouping transformation module splits the input multivariate time series data into univariate time series data, and then inputs each univariate time series data into different linear layers for linear transformation, and then performs a splicing operation on the output data of each linear layer. The spliced multivariate time series data features are then input into the channel adaptive graph attention module.

[0115] The channel adaptive graph attention module processes the data output by the grouped linear transformation module as follows:

[0116] ①Calculate channel characteristics:

[0117]

[0118] Among them, X :,t,: Represents the data of all batches and all channels of the input channel adaptive graph attention module at the t-th time step, and L represents the length of the input data.

[0119] ②Build a dynamic adjacency matrix between channels:

[0120] Perform linear transformation on the channel features and generate a normalized adjacency matrix A as the channel interaction weight:

[0121]

[0122] in, is the weight matrix, b is the bias vector, and the matrix is normalized using Softmax.

[0123] ③ Information fusion of channel association relationships:

[0124] The dynamic adjacency matrix A is used to weight the data X of the input channel adaptive graph attention module:

[0125] X′=X·A (8)

[0126] in, Where B represents the batch size of the data, L represents the length of the data, and J represents the number of channels of the data; The weighted result obtained by multiplying the two Same shape as X.

[0127] ④Residual feature fusion:

[0128] X out =X′+X (9)

[0129] Fusion DataX out Enter the LeakyRelu activation function layer;

[0130] In the first stage of the adversarial training process, the total loss L is introduced latent Acting on the generator and introducing the total loss L R3 Acts on both the generator and the discriminator; in the second stage of the anti-training training process, only the total loss L is introduced R3 It works on both the generator and the discriminator.

[0131] Total loss L latent The expression is as follows:

[0132] L latent =L cst +λL spec (10)

[0133] Among them, L cst is the consistency loss, L spec is spectrum loss; is a weight hyperparameter used to balance the contribution of consistency and spectral loss.

[0134] Consistency loss L cst Used to constrain the distribution between the data generated by the generator and the real data, L cst The expression is as follows:

[0135]

[0136] in, For the generator Fake samples generated for input, is the real data, which belongs to category i and contains all channels;

[0137]

[0138] in, is the category i, channel j and belongs to the kth i,j Based on this approach, the generator can find the corresponding cluster in the real dataset according to the category to which the GMN belongs and the components of the corresponding Gaussian variables.

[0139] In model training, although GMN is used as an implicit conditional input for the generator, the weights of the GAN model are randomly initialized, so the generator cannot quickly learn the distribution of real data. i,j The different components in represent different patterns in the category, i.e., subcategories. However, due to the lack of explicit generation constraints on subcategory-level data in the design of GMN-GAN, the generator cannot correctly learn the correspondence between the latent space (i.e., noise space) and the subcategory pattern. Therefore, the data generated by the generator through the GMN corresponding to the specified subcategory cannot match the corresponding real data, that is, the mapping of the generator's latent space to the sample space at the subcategory level is chaotic, which in turn affects the pattern correspondence of the generated data. In order to solve the above problem, the present invention has a total loss L latent Introducing consistency loss L cst , which is achieved by dividing the output of the generator by the noise in its input latent space. The generator is encouraged to learn the accurate mapping relationship between the GMN latent space and the sample space, so that the subclasses in the latent space can be accurately mapped to the corresponding subclasses in the sample space, thereby maintaining their distribution consistency.

[0140] Due to L cst The data distribution is matched mainly by minimizing the Euclidean distance between the synthetic samples and the real samples. In practical applications, this may cause the generator to generate more noise, affecting the quality of the generated data. Therefore, based on the consistency loss, the present invention reduces the total loss L latent Spectral loss is further introduced to reduce the noise interference in the generated data. spec It is used to constrain the spectrum similarity between the data generated by the generator and the real data, L spec The expression is as follows:

[0141]

[0142] Here, STFT(·) represents the short-time Fourier transform operation.

[0143] During the training process, the total loss L cst The backpropagation is passed to the generator to constrain the distribution consistency between the latent space and the generated sample space.

[0144] The above-mentioned introduction of distribution consistency constraints in the GMN-GAN generator training process improves the mapping consistency between the latent space and the sample space, and effectively reduces the noise interference in the generated data, thereby improving the quality of data generation.

[0145] Traditional GAN training often faces the problems of training instability (such as gradient disappearance and explosion) and mode collapse. To solve this problem, the present invention reduces the total loss L R3 RpGAN loss and zero-center gradient penalty are introduced in

[15] .

[0146] Total loss L R3 The expression is as follows:

[0147]

[0148] Among them, L RpGAN is the RpGAN loss, L R1 and L R2 is the gradient penalty, γ is a hyperparameter used to control the gradient penalty terms LR1 and L R2 strength.

[0149] L RpGAN The loss is described as follows:

[0150]

[0151] Among them, p z (z) is the distribution of noise z, p data (x) is the distribution of the real sample x, D(G(z)) is the discriminant output of the discriminator for the generated sample G(z), D(x) is the discriminant output of the discriminator for the real sample x, and function f is the Softplus function. L RpGAN This loss enables the discriminator to judge the authenticity of generated samples based on their relative authenticity to real samples, rather than focusing solely on the degree of classification between generated samples and real samples. Measuring the authenticity of each generated sample relative to real samples, this coupling of real and generated data helps maintain the decision boundary within the distribution of real data, thereby mitigating mode collapse.

[0152] Although RpGAN alleviates the mode collapse problem by introducing relative pairing loss, another major challenge in GAN training is the gradient instability of the discriminator, which may lead to gradient vanishing or gradient explosion, thus affecting the training of the generator. In order to solve the gradient instability problem, the present invention introduces a gradient instability in the total loss L R3 Two gradient penalty terms are introduced.

[0153] Gradient penalty term L R1 The description is as follows:

[0154]

[0155] Where L is the length of the input time series; d is the number of sensors, that is, the total number of variable dimensions of the time series; is the partial derivative of the discriminator with respect to the value of the j-th variable at the l-th time step, that is, the gradient of the data point.

[0156] Gradient penalty term L R2 The description is as follows:

[0157]

[0158] in, Represents the partial derivative of the discriminator with respect to the value of the j-th variable at the l-th time step in the generated data, that is, the gradient of the data point.

[0159] Gradient penalty term L R1 Acting on real data, the gradient penalty term L R2 Acting on the generated data. During the training process, the total loss L R3 Passed back to the generator and discriminator, if the generator G can already generate data with real distribution, the discriminator D will not push the generator G away from the optimal state; by constraining the gradient norm of the discriminator, the oscillation generated during training can be effectively reduced.

[0160] IV) The Gaussian mixture model finds the corresponding Gaussian components according to the labels and indices of the samples in the electronic nose dataset, samples the Gaussian components to obtain Gaussian mixture noise, and inputs the Gaussian mixture noise into the generative adversarial network trained in step III), and the generator generates enhanced data.

[0161] The following experiments and analysis are conducted on the generative adversarial network method for enhancing multivariate time series data of electronic nose proposed in the present invention. The model architecture of the generative adversarial network method for enhancing multivariate time series data of electronic nose proposed in the present invention is as follows: Figure 1 As shown, it is named MTS-GMN-GAN model.

[0162] Experimental setup

[0163] The electronic nose dataset used in all the following experiments is the lung cancer multivariate time series dataset, referred to as the LC-MTS dataset. It is constructed using data collected by 22 gas sensors as shown in Table 1. Each sample contains 22 channels of time series data, and the length is 360.

[0164] Table 1 Information of gas sensors in the sensor array

[0165]

[0166]

[0167] All experiments were implemented using the PyTorch deep learning framework in Python and run on an NVIDIA TITANXp graphics card (3840 cores, 12GB of global memory). The hyperparameter settings used during training of the MTS-GMN-GAN model are shown in Table 2.

[0168] Table 2 Hyperparameter settings of the MTS-GMN-GAN model

[0169]

[0170] Sensor array optimization evaluation

[0171] When optimizing the sensor array, the training set (X tr ,Y tr ) is set aside as a validation set to evaluate the performance of the model during the array optimization process. The remaining data is used to execute the LDA-based sensor array optimization algorithm. Specifically, the weight vector W of each sensor is calculated by formula (1), and all sensors are sorted in descending order from large to small according to their weights. Unlike the traditional direct selection method or stepwise selection method, an incremental optimization strategy is adopted, that is, starting from an empty set, a new sensor is selected in each iteration to be added to the current optimal combination, and the classification model is trained based on the sensor subset. In addition, in the selection of the classification model, because the support vector machine (SVM) has a lower time complexity in calculation, the present invention selects SVM as the evaluation classifier.

[0172] To evaluate the classification performance after each round of sensor addition, the classification accuracy was calculated on the validation set after each sensor was added, and its changing trend was recorded. When the validation set accuracy first reached its maximum value, the number of sensors at that time was considered the optimal value and the result of sensor array optimization. Figure 7 The changing trend of the accuracy of the lung cancer dataset validation set under different numbers of sensors is shown. Figure 7 The experimental results presented show that the validation set accuracy reaches its maximum for the first time when the number of sensors is 8 (marked by the red dots in the figure). Therefore, this paper selected 8 sensors as the optimized sensor array configuration. In subsequent data generation and downstream classification tasks, the experiment will only use data from these 8 sensors as input. The final 8 sensor numbers and their corresponding model information are shown in Table 3.

[0173] Table 3 Selected sensors and their information

[0174]

[0175] Tables 4 and 5 compare the performance of different models in classification tasks before and after sensor array optimization. The evaluated models include traditional machine learning methods (SVM, MiniROCKET, and ROCKET) as well as various deep learning models (FCN, InceptionTime, and LSTM-FCN). Performance metrics include sensitivity (Sens), specificity (Spec), area under the receiver operating characteristic curve (AUC), accuracy (Acc), precision (Prec), and F1 score (F1).

[0176] Table 4 Comparison of classification performance of different models before sensor array optimization

[0177]

[0178] Table 5 Comparison of classification performance of different models after sensor array optimization

[0179]

[0180] According to the experimental results in Table 4, before optimization, LSTM-FCN achieved the highest sensitivity (94.55%), but its specificity was lower (56.67%). MiniROCKET performed well in overall classification performance, with accuracy (84.71%), AUC (81.36%), and F1 score (88.69%) all outperforming most models. The experimental results in Table 5 reflect the changes in classification performance after sensor array optimization. After optimization, with the exception of SVM, which maintained its performance, the performance of other models improved to varying degrees. MiniROCKET's sensitivity increased to 100%, AUC to 83.33%, accuracy to 88.24%, and F1 score to 91.67%, demonstrating significant performance improvements. Other models also showed some improvement. The AUC of FCN reached 81.21% and the accuracy increased to 83.53%; the AUC of InceptionTime increased to 75.15% and the accuracy increased to 77.65%; the AUC and accuracy of LSTM-FCN increased to 81.21% and 83.53% respectively.

[0181] Overall, sensor array optimization significantly improved the classification performance of most models, particularly MiniROCKET and FCN, which exhibited superior performance across multiple metrics. This demonstrates that the optimized sensor array effectively reduces sensor data redundancy, helping classification models extract more discriminative features, thereby enhancing their recognition capabilities. Furthermore, reduced dimensionality not only improves the training efficiency of subsequent generative models but also reduces their modeling complexity, further improving their generalization capabilities.

[0182] Model comparison experiment

[0183] To further evaluate the quality of data generated by the MTS-GMN-GAN model, we compared it with other models synthesized using a sensor array-optimized dataset: TimeGAN, Sig-WGAN, TimeVAE, and Biodiffusion. All models synthesized data using a sample size comparable to the training set, and quality was assessed using a combination of subjective and objective evaluations. Furthermore, we compared the training time, inference time, parameter size, and computational complexity of the different models.

[0184] There are two types of subjective indicators: the first is to directly draw time series curves of real data and synthetic data. Since each MTS sample contains multiple channels, it is necessary to conduct intuitive evaluation from two perspectives. First, some real MTS samples and MTS samples synthesized by different models are randomly selected and compared visually on a sample-by-sample basis (e.g. Figure 8 As shown), observe the similarity between the data synthesized by different models and the real data. Secondly, the data of different channels of the real data are compared with the data of different channels of the data synthesized by different models in a visual way, so as to observe the pattern coverage of the data synthesized by the model in different channels and evaluate its generation quality (as shown). Figure 9 The other method is to reduce the dimension of the real data and the synthetic data to a 2D plane for visualization. The dimensionality reduction methods used include PCA (such as Figure 10 ) and t-SNE methods (as shown Figure 11 shown).

[0185] Figure 8 The following figure visualizes the MTS samples generated by different models. Each model displays two columns of data, each containing three samples: the first column is for healthy individuals, and the second column is for individuals with lung cancer. It is easy to see that TimeGAN, TimeVAE, and MTS-GMN-GAN can generate data with a high degree of similarity to real data. Among them, TimeGAN has a relatively simple model, TimeVAE generates some data that differs significantly from real data, and the MTS-GMN-GAN proposed in this paper achieves the best overall performance.

[0186] exist Figure 9 The results shown show the data distribution of real data and synthetic data from other models in each channel. The red curve represents the curve drawn by the real data, and the blue curve represents the curve drawn by the synthetic data. Figure 9The eight channels (Channel 0 to 7) in the dataset correspond to the indices (Index 1, 4, 7, 10, 11, 15, 22) in Table 4. The visualization results show that most models struggle to cover all the patterns across different channels. Of particular note is TimeVAE, which, while offering the broadest coverage, also exhibits the highest degree of deviation. The samples synthesized by the proposed MTS-GMN-GAN can better cover the different patterns in the real data, with minimal deviation from the real data, resulting in the best pattern coverage.

[0187] Figure 10 and Figure 11 In the results shown, red dots represent sample points after dimensionality reduction of real data, and blue dots represent sample points after dimensionality reduction of synthetic data. Experimental results show that the distribution of MTS-GMN-GAN is closest to that of real data in PCA dimensionality reduction results; and in t-SNE dimensionality reduction results, MTS-GMN-GAN has the most consistent distribution with real data.

[0188] Table 6 below shows the comparative results of the quality assessment of data synthesized by the MTS-GMN-GAN model proposed in this invention and other methods. The data in the first row represents the evaluation results using real data, which serves as a reference for data authenticity. As can be seen from the table, the MTS-GMN-GAN proposed in this invention achieved the best performance in terms of FID, discriminant score, and predictive score, and was closest to the scores of real data. This shows that the model expanded by the present invention based on UTS-GMN-GAN can fully utilize the advantages of UTS-GMN-GAN and generate high-quality lung cancer MTS data on the LC-MTS dataset.

[0189] Table 7 further shows the computational resources of different generative models on the LC-MTS dataset, including key metrics such as training time, inference time, model parameter count, and computational effort, to comprehensively evaluate the practical application value of each method. Based on the experimental results, the proposed MTS-GMN-GAN model has the least computational effort, suboptimal inference time and parameter count, and relatively short training time, giving it a certain advantage over the compared models.

[0190] Table 6 Performance comparison of different time series generation models (MTS)

[0191]

[0192] Table 7 Comparison of computing resources of different time series generation models (MTS)

[0193]

[0194] Downstream task evaluation experiments

[0195] The experiment first evaluated the impact of five different generative models (TimeGAN, Sig-WGAN, TimeVAE, Biodiffusion, and the MTS-GMN-GAN (Ours) proposed in this paper) on the classification performance of six classification models (SVM, MiniROCKET, ROCKET, FCN, InceptionTime, and LSTM-FCN) after being applied to data augmentation. The experimental results are shown in Figure 2. Figure 12 Each subgraph corresponds to the evaluation result of a classification model. The horizontal axis represents different generation methods, and the vertical axis represents the change in classification performance. The evaluation indicators include sensitivity (Sens), specificity (Spec), AUC, accuracy (Acc), precision (Prec), and F1 score (F1).

[0196] according to Figure 12 From the experimental results shown, it can be found that there are significant differences in the effects of using different generative models for data augmentation on the performance of the classification model. The classification performance of SVM is significantly improved when Sig-WGAN is used for data augmentation, while the other generative models fail to bring significant improvements, and even lead to performance degradation in some cases. After the MTS-GMN-GAN proposed in the present invention is used for data augmentation, the classification effect of SVM remains basically unchanged. MiniROCKET only experienced a slight performance degradation when using the TimeGAN model for data augmentation, while there were improvements to varying degrees when using other generative models. Among them, the model MTS-GMN-GAN proposed in the present invention had the best overall enhancement effect. After using different generative models for data augmentation, ROCKET showed a consistent improvement effect except for the slightly weaker improvement effect of BioDiffusion. Among the deep learning classification models such as FCN, InceptionTime and LSTM-FCN, MTS-GMN-GAN maintained the best classification performance improvement effect, while the improvement of the other models was relatively limited, and some even led to a decline in classification performance. Overall, MTS-GMN-GAN, as a data augmentation method, has improved the overall performance of the classification model on the MTS data of the lung cancer electronic nose, especially the performance on the deep learning classification model.

[0197] In addition, considering that using different amounts of generated data for data augmentation may lead to different performance of downstream classification models, the present invention further explores the impact of different data generation multipliers on classification performance. The experimental results are as follows: Figure 13The vertical axis of the figure represents classification performance indicators (Sens, Spec, AUC, Acc, Prec, and F1), while the horizontal axis represents different data generation multiples. The evaluation objects still include six classification models, including traditional machine learning methods (SVM, MiniROCKET, and ROCKET), as well as deep learning methods (FCN, InceptionTime, and LSTM-FCN).

[0198] Experimental results show that the introduction of MTS-GMN-GAN generated data has a positive impact on the performance of most classification models. When the data generation multiple is moderate (such as 1 to 4 times), it can effectively improve the performance of most classification models, which shows that the data generated by MTS-GMN-GAN helps to enhance the discrimination ability of the model. Among them, deep learning models (such as FCN, InceptionTime and LSTM-FCN) benefit more significantly, and their performance is particularly outstanding in indicators such as AUC and F1 scores that measure overall classification performance. Specifically, the introduction of MTS-GMN-GAN generated data has certain differences in the impact on different classification models. SVM ( Figure 13 (a) When the data generation multiplier is 4, all indicators except Spec are improved and remain stable. This may indicate that the statistical characteristics of SVM, as the only model that relies on manual features (based on the maximum value of each sensor channel) for classification, do not change significantly after data augmentation, so the improvement in classification performance is relatively limited. MiniROCKET ( Figure 13 (b)) The performance improved when 1x generated data was introduced, but as the number of generated data increased, its classification performance fluctuated to a certain extent, and when 6x generated data was introduced, its classification performance began to show a significant downward trend. This may be attributed to the excessive amount of generated data introducing noise, or causing an increase in the proportion of low-quality samples, which in turn affected the classification effect. Figure 13(c) As the number of generated data increases, the overall performance changes more steadily than MiniROCKET, achieving optimal classification performance at 1, 4, 8, and 10x augmentation. This indicates that ROCKET is more robust than MiniROCKET and, to a certain extent, better adaptable to low-quality samples in generated data. Both FCN and InceptionTime achieve optimal performance at 2x data augmentation, but FCN's overall performance trend increases and then stabilizes, while InceptionTime exhibits greater fluctuations. This result demonstrates that even within deep learning models, classification models with different network architectures exhibit varying adaptability to generated data. LSTM-FCN exhibits similar performance improvement trends to FCN, showing an overall steady increase and then plateauing, and its stability is superior to FCN. This may indicate that LSTM-FCN is more adaptable to data generated by MTS-GMN-GAN.

[0199] In summary, MTS-GMN-GAN, as a data augmentation method, can improve the performance of classification models when introducing an appropriate amount of generated data (approximately 1 to 4 times), with a particularly significant enhancement effect on deep learning models. For most models, generating approximately 4 times the amount of data can achieve good performance. However, introducing excessive generated data (approximately 6 times the amount) may introduce more noise or low-quality samples, resulting in performance degradation for some classification models. Therefore, in practical data augmentation applications, it is necessary not only to comprehensively consider the balance between the quality and quantity of generated data, but also to optimize the data augmentation strategy based on the characteristics of different classification models to ensure optimal performance of the classification model.

[0200] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions of the present invention, and all such changes should be included in the scope of the claims of the present invention.

Claims

1. A generative adversarial network method for electronic nose multivariate time series data enhancement, characterized by: The following steps are involved: I) For the multivariate time series data collected by the gas sensor array of the electronic nose, one gas sensor corresponds to one data channel. The linear discriminant analysis method is used to remove redundant channel data. The linear discriminant function is as follows: g(F)=W T F (1) Where W=[w1,w2,…,w j ,…,w d ] T is a weight vector whose elements are calculated from the intra-class scatter matrix and the inter-class scatter matrix; F = f1, f2, ..., f j ,…,f d ] T is the eigenvector of the gas sensor array, j∈{1,2,…,d} represents the channel number, d is the number of sensors, f j represents the characteristic value of the jth gas sensor, f j Calculated from the maximum signal value of the jth channel: Among them, the function g(X k ,j)Return sample X k The j-th dimension time series S j , max(·) returns S j The maximum value of all elements in ; II) After removing redundant channel data, a Gaussian mixture model is fitted for the data from different channels in the same category of data. Each Gaussian mixture model is expressed as: Among them, V i,j is the Gaussian mixture model corresponding to the jth channel of the i-th category, i∈{1,2,…,C} represents the category number, k i,j Indicates V i,j kth i,j Gaussian components, K i,j Represents the total number of Gaussian components corresponding to the j-th channel of category i, π i,j,k The kth channel corresponding to the jth channel of category i i,j The weights of the Gaussian components, The representative mean is The covariance is Gaussian distribution; Each sample in a multivariate time series dataset is assigned an index pair (i, j, k i ), used for noise generation of different patterns within a class; III) performing adversarial training on a generative adversarial network, which includes a generator and a discriminator. The adversarial training process is divided into two stages, and each stage of adversarial training includes: 1) Gaussian mixture model is based on the label and index of the sample (i, j, k i ) Find the corresponding Gaussian component k i,j , for Gaussian component k i,j Sampling Gaussian mixed noise, in, is Gaussian mixture noise; For each sample of category i, its noise latent space vector is expressed as: where k i =[k i,1 ,k i,2 ,…,k i, d] represents the combination of Gaussian component numbers of the Gaussian mixture model corresponding to different channels in the sample of category i; 2) Mix the Gaussian noise As the input noise of the generator, the generator generates fake samples; 3) The fake samples generated by the generator and the real samples from the real data are input into the discriminator, which judges the fake samples and the real samples to obtain the discrimination results; The generator and the discriminator both include several group transformation modules containing channel adaptive graph attention, and the group transformation modules containing channel adaptive graph attention include a channel group transformation module that performs linear transformation on each pass data, a channel adaptive graph attention module connected to the output of the channel group transformation module, and a LeakyRelu activation function layer connected to the output of the channel adaptive graph attention module; The channel grouping transformation module splits the input multivariate time series data into univariate time series data, and then inputs each univariate time series data into different linear layers for linear transformation, and then splices the output data of each linear layer. The spliced multivariate time series data features are then input into the channel adaptive graph attention module; The channel adaptive graph attention module processes the data output by the grouped linear transformation module as follows: ①Calculate channel characteristics: Among them, X :,t,: Represents the data of all batches and all channels of the input channel adaptive graph attention module at the t-th time step, and L represents the length of the input data; ②Build a dynamic adjacency matrix between channels: Perform linear transformation on the channel features and generate the normalized adjacency matrix A as the channel interaction weight: in, is the weight matrix, b is the bias vector, and the matrix is normalized using Softmax; ③ Information fusion of channel association relationships: The dynamic adjacency matrix A is used to weight the data X of the input channel adaptive graph attention module: X′=X·A (8) in, Where B represents the batch size of the data, L represents the length of the data, and J represents the number of channels of the data; The weighted result obtained by multiplying the two Same shape as X; ④Residual feature fusion: X out =X′+X (9) Fusion DataX out Enter the LeakyRelu activation function layer; In the first stage of the adversarial training process, the total loss L is introduced latent Acting on the generator and introducing the total loss L R3 Acts on both the generator and the discriminator; in the second stage of the anti-training training process, only the total loss L is introduced R3 Acts on both the generator and the discriminator; Total loss L latent The expression is as follows: THE latent =L cst +λL spec (10) Among them, L cst is the consistency loss, L spec is spectrum loss; is a weight hyperparameter used to balance the contribution of consistency and spectral loss; Consistency loss L cst Used to constrain the distribution between the data generated by the generator and the real data, L cst The expression is as follows: in, For the generator Fake samples generated for input; is the real data, which belongs to category i and contains all channels; in, is the category i, channel j and belongs to the kth i,j True samples of clusters, j∈{1,2,…,d}; Spectrum loss L spec It is used to constrain the spectrum similarity between the data generated by the generator and the real data, L spec The expression is as follows: Where STFT(·) represents the short-time Fourier transform operation; During the training process, the total loss L cst Backward pass to the generator to constrain the distribution consistency between the latent space and the generated sample space; Total loss L R3 The expression is as follows: L RpGAN The loss is described as follows: Among them, p z (z) is the distribution of noise z, p data (x) is the distribution of the real sample x, D(G(z)) is the discriminant output of the discriminator for the generated sample G(z), D(x) is the discriminant output of the discriminator for the real sample x, and function f is the Softplus function; Gradient penalty term L R1 The description is as follows: Where L is the length of the input time series; d is the number of sensors, that is, the total number of variable dimensions of the time series; is the partial derivative of the discriminator with respect to the value of the j-th variable at the l-th time step, that is, the gradient of the data point; Gradient penalty term L R2 The description is as follows: in, Represents the partial derivative of the discriminator with respect to the value of the j-th variable at the l-th time step in the generated data, that is, the gradient of the data point; γ is a hyperparameter used to control the gradient penalty term L R1 and L R2 Strength; During the training process, the total loss L R3 Backward pass to the generator and discriminator; IV) The Gaussian mixture model finds the corresponding Gaussian components according to the labels and indices of the samples in the electronic nose dataset, samples the Gaussian components to obtain Gaussian mixture noise, and inputs the Gaussian mixture noise into the generative adversarial network trained in step III), and the generator generates enhanced data.

2. The generative adversarial network method for electronic nose multivariate time series data enhancement according to claim 1, characterized in that: The network structure of the generator in step III) includes four sequentially connected group transformation modules containing channel adaptive graph attention and a Tanh activation function layer connected to the output of the fourth group transformation module containing channel adaptive graph attention; the network structure of the discriminator in step III) includes three group transformation modules containing channel adaptive graph attention, a first convolutional layer connected to the output of the third group transformation module containing channel adaptive graph attention, a first LeakyRelu activation function layer connected to the output of the first convolutional layer, a second convolutional layer connected to the output of the first LeakyRelu activation function layer, a second LeakyRelu activation function layer connected to the output of the second convolutional layer, and a Sigmoid function layer connected to the output of the second LeakyRelu activation function layer.

Images