Electroencephalogram signal generation method and system fusing time-frequency domain and differential entropy features

Abstract

The invention discloses an electroencephalogram signal generation method and system fusing time-frequency domain and differential entropy features. And adding the original electroencephalogram signals into a VAE encoder to obtain hidden space distribution. In a VAE decoder, standard Gaussian noise and hidden space noise are added to respectively obtain generated electroencephalogram and reconstructed electroencephalogram. And inputting the generated electroencephalogram, the reconstructed electroencephalogram and the original electroencephalogram into a discriminator to judge authenticity. In the training process, KL divergence loss is adopted to train VAE encoder parameters, reconstruction loss and similarity loss are adopted to train VAE decoder parameters, and cross entropy loss and a gradient penalty function are adopted to train discriminator parameters. In the training process, the VAE decoder continuously generates more real electroencephalogram signals. Finally, the VAE decoder and the discriminator are balanced, and the VAE decoder achieves the ability of amplifying the electroencephalogram signal data set. According to the method, the electroencephalogram data set can be expanded by generating more real electroencephalogram data, so that the generalization ability and performance of a network model are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of data generation, and specifically relates to an electroencephalogram (EEG) signal generation method and system integrating time-frequency domain and differential entropy features, namely, an EEG signal data amplification method. Background Art

[0002] EEG signals refer to bioelectric signals generated by information transmission between neurons when the brain is active. These signals are closely related to the activity state of the brain and can reflect the different functional states of the brain. With the development of EEG signal analysis technology, applications such as epilepsy detection, sleep staging and emotion recognition based on EEG signals have received widespread attention. However, due to factors such as experimental equipment, experimental paradigms, acquisition time, and individual differences, the current mainstream EEG data sets are usually small in size, making them difficult to promote in practical applications. Therefore, increasing the collective data volume, that is, generating data, has become an important means to improve the application of EEG signals in practical problems.

[0003] The current mainstream data generation method is based on Generative Adversarial Network (GAN). These methods construct generative models and discriminative models, and enable them to have self-learning capabilities through continuous adversarial training, thereby generating data with similar distribution. However, there is a problem of unstable generation process. Another commonly used generative model is Variational Autoencoder (VAE), which obtains the hidden layer encoding and probabilistic expression of the data through the encoding and decoding structure, and then decodes high-quality generated data. However, how to design the encoding and decoding structure according to the characteristics of EEG data and generate more data to assist downstream tasks is the key to generating high-quality EEG data.

[0004] Therefore, the present invention combines the time-frequency domain and differential entropy characteristics of EEG signals into the VAE-GAN structure of the dual encoder to generate high-quality EEG signals that conform to the data distribution characteristics, so as to broaden its application in downstream tasks such as epilepsy detection, sleep staging and emotion recognition. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a method and system for generating EEG signals that integrate time-frequency domain and differential entropy features. The method draws on the idea of ​​generative adversarial GAN ​​networks, uses a dual encoder that integrates time-frequency domain and differential entropy features to obtain the distribution of EEG data in latent space, and uses a VAE decoder as a generator to decode random sampling noise to generate data that conforms to the distribution of real EEG signals; a classification network containing a residual structure is used as a discriminator to identify true and false EEG signal samples. During the training process, the model training process is stabilized by measures such as gradient penalty, thereby improving the stability of the network model.

[0006] The technical solutions specifically adopted in the present invention are as follows:

[0007] In a first aspect, the present invention provides a method for generating an EEG signal by integrating time-frequency domain and differential entropy features, comprising the following steps:

[0008] Step 1: Dataset division: The original EEG data is divided into training set and test set in proportion.

[0009] Step 2: Extract time-frequency features: For the original EEG signals in the training set, perform continuous wavelet transform (CWT) to obtain the time-frequency (TF) features of the EEG signals. The TF features are complex numbers, including real data Xr and imaginary data Xi of the same dimension.

[0010] Step 3: Differential entropy feature extraction: For the original EEG signals in the training set, the DE features of the EEG signals are obtained by differential entropy (DE) calculation.

[0011] Step 4: Establish and train the EEG generation model; specifically:

[0012] 4.1 Construct a VAE dual encoder network and encode it according to the time-frequency features and differential entropy features to obtain the latent space representation; specifically:

[0013] The VAE dual encoder network includes a time-frequency encoder, a differential entropy encoder, and a latent space layer; the time-frequency encoder is used to calculate the first latent space distribution of the time-frequency features. The differential entropy encoder is used to calculate the second latent space distribution of time-frequency features. Finally, the hidden space layer is used to distribute the first hidden space and the second latent space distribution Splicing to get the final latent space distribution It is the random noise in latent space.

[0014] The time-frequency encoder includes three layers of complex convolutional neural networks and a fully connected layer connected in series. The input of each layer in the three-layer complex convolutional neural network is composed of real data Xr and imaginary data Xi of the same dimension; the fully connected layer is used to take the absolute value of the output of the three-layer complex convolutional neural network and then reduce the dimension.

[0015] The differential entropy encoder comprises three one-dimensional convolutional layers and a fully connected layer connected in series. The fully connected layer is used to perform dimensionality reduction processing on the outputs of the three one-dimensional convolutional layers.

[0016] 4.2 Construct a VAE decoder network; input standard Gaussian noise into the VAE decoder network to obtain generated EEG, and input the latent space representation into the VAE decoder network to obtain reconstructed EEG;

[0017] The VAE decoder network uses a convolutional network with a self-attention mechanism to reconstruct and generate EEG data. The reconstruction task is to distribute The random noise sampled from the prior distribution is input into the decoder to obtain the reconstructed data. The generation task is to sample random noise from the prior distribution, that is, the standard Gaussian distribution, and input it into the decoder to obtain the generated data.

[0018] The VAE decoder network architecture includes a first deconvolution layer, a second deconvolution layer, a first self-attention layer and two convolution layers connected in series.

[0019] 4.3 Construct a discriminator with a residual network to identify the original EEG in the training set in step 1 and the EEG generated and reconstructed in step 5;

[0020] The discriminator with residual network marks the original EEG signal as a true sample, and the generated EEG and reconstructed EEG as a false sample. Its network structure consists of three one-dimensional convolutional layers, one self-attention mechanism layer and two fully connected layers.

[0021] The discriminator with residual network includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a second self-attention layer and two fully connected layers connected in series in sequence; the first convolutional layer, the second convolutional layer and the third convolutional layer have the same structure, and all include a one-dimensional convolutional layer and an average pooling layer connected in series. A residual connection is also provided between the first convolutional layer and the second convolutional layer, and a residual connection is also provided between the second convolutional layer and the third convolutional layer; the first convolutional layer and the second convolutional layer each perform convolution operations on the input of the corresponding scale, and directly add the input features of the current convolutional layer to the output features as a residual connection method.

[0022] 4.4 Repeat the training process of steps 4.1-4.3 until the VAE decoder network and the discriminator with residual network reach a balance, and the VAE decoder network reaches the ability to expand the data set;

[0023] Loss and parameter optimization of EEG generation model during training: For the mean and variance in the latent space obtained by the VAE encoder network, the KL divergence loss L is used KL (P||Q) approximates the prior distribution, i.e., the standard normal distribution, and trains the network parameters θ1 of the VAE encoder network.

[0024] KL divergence loss L KL (P||Q):

[0025]

[0026] Where P represents the latent space distribution Q represents the standard Gaussian distribution Q~N(0,1), M represents the dimension of the latent space, μ i and σ iis the i-th component of the mean and variance of the posterior distribution P;

[0027] For the reconstructed data obtained by the VAE decoder network, similarity loss is introduced Calculate the network loss and compare the reconstructed data with the real data to obtain the reconstruction loss L recon , train the network parameters θ2 of the VAE decoder network.

[0028] Similarity loss

[0029]

[0030] Where P represents the latent space distribution, z r represents random noise sampled from the latent space distribution, G is the VAE decoder network, and D is the discriminator; Represents sampling noise z from the latent space r The expected value obtained by the decoder and the discriminator.

[0031] Reconstruction loss L recon :

[0032]

[0033] Where MSE is the mean square loss function, P data(x) represents the original data distribution; Represents the original EEG x and the reconstructed EEG G(z r ) is the expected mean square error between the two.

[0034] The true and false discrimination results obtained by the discriminator with residual network are calculated using the cross entropy loss function. The generated data and real data are input into the discriminator to judge the authenticity of the input signal and train the discriminator. The gradient penalty loss is also added to the parameters of the discriminator network itself to train the network parameters θ3 of the discriminator with residual network.

[0035] Cross entropy loss function:

[0036]

[0037] Where z represents standard Gaussian noise, λ represents an adjustable hyperparameter, θ3 represents the discriminator network parameter; N(0,1) represents the standard normal distribution; ∥·∥2 represents the L2 norm.

[0038] Specifically, in each adversarial training, the original data passes through the discriminator to obtain the discrimination result loss, and then random noise is sampled from the standard Gaussian distribution and put into the decoder to obtain the generated data, which is passed through the discriminator to obtain the discrimination loss. The original data discrimination result loss, the generated data discrimination result loss and the gradient penalty loss of the discriminator network are back-propagated to train the discriminator to obtain the parameter θ3.

[0039] After the discriminator is trained, the original data is compressed by the VAE encoder to obtain the latent space distribution The KL divergence calculation of the latent space distribution and the standard Gaussian distribution is used to obtain the loss L KL (P||Q), back propagate L KL (P||Q) loss trains the VAE encoder to obtain the network parameter θ1. Random noise is sampled from the latent space and input into the VAE decoder to obtain the reconstructed data. The difference between the reconstructed data and the real data is calculated to obtain the reconstruction loss L recon , and input the reconstructed data into the discriminator to obtain the similarity loss Back-transmission L recon , The loss trains the generator, i.e. the decoder, to obtain the parameter θ2. Repeating the above steps, the generator and the discriminator finally reach a balance, and the generator is able to expand the data set.

[0040] Step 5. For the random noise sampled from the standard Gaussian distribution, use the optimized network parameter θ2 and the trained VAE decoder network to obtain the optimal generated EEG data, that is, the expansion of random noise generation; test and verify the optimal generated EEG data and the original EEG data of the test set; finally, use the trained and tested VAE decoder network to realize EEG signal amplification.

[0041] In a second aspect, the present invention provides an electroencephalogram signal generating system, comprising:

[0042] Data acquisition module, used to extract the time-frequency characteristics and differential entropy characteristics of the original EEG signal;

[0043] The EEG generation module realizes EEG signal amplification based on the time-frequency characteristics and differential entropy characteristics of the original EEG signal.

[0044] In a third aspect, the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the method.

[0045] In a fourth aspect, the present invention provides a machine-readable storage medium storing machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the method.

[0046] Beneficial effects of the present invention:

[0047] The present invention proposes to use a VAE-GAN network model that integrates time-frequency domain and differential entropy as an EEG generation model. The model combines complex neural networks and multi-domain feature fusion technology and inherits the idea of ​​WGAN-GP, that is, the norm of the discriminator is constrained by gradient penalty. The model uses TF features and DE features extracted from the original EEG as encoder input, and complex convolution layers and deep convolution layers as core components of the encoder to reconstruct the EEG signal data distribution; the decoder part is composed of a deconvolution layer and a self-attention mechanism. The self-attention layer can learn the relationship coefficients between each channel. Therefore, the use of the self-attention layer can capture a wider range of feature information and construct more realistic EEG data. The discriminator uses a one-dimensional convolution layer with an average pooling layer and a self-attention mechanism layer to identify the authenticity of the data. Each one-dimensional convolution layer has an additional residual connection, which can quickly converge the model and distinguish true and false data. The training of the generator is guided by the error of the discrimination result. Based on this, the actual effects produced by the present invention include:

[0048] Data augmentation: By generating more realistic EEG signal data, the EEG signal dataset can be augmented, providing more training samples and improving the classification accuracy of downstream classification tasks.

[0049] Model improvement: The complex network is used as a component of the encoder to process the complex features obtained after the EEG is transformed through time-frequency transformation. The dual encoder architecture combines multiple EEG features to improve the model's ability to model EEG signal data, thereby generating more realistic and accurate EEG signal data.

[0050] In summary, the VAE-GAN EEG generation method that integrates time-frequency domain and differential entropy features proposed in the present invention can increase the amount of EEG signal data, increase the size of the EEG signal data set, enhance the performance and robustness of the model, and help further promote the development of EEG signal data analysis and application. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a framework flow chart of the method of the present invention.

[0052] Figure 2 The generation model and training flow chart of the method of the present invention.

[0053] Figure 3A This is a brain topography map of the test data in the embodiment of the present invention.

[0054] Figure 3B This is a brain topography map that generates data in an embodiment of the present invention.

[0055] Figure 4This is a T-SNE dimensionality reduction algorithm diagram of generated data and test data in an embodiment of the present invention, wherein the test data, i.e., the real samples, are in the form of strips, and the generated samples are in the form of dots. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present invention more clear, this embodiment uses the SLEEPEDFX public data set combined with the attached Figure 1 , 2 , 3 and the public dataset SEED collection Figure 4 The method for generating an EEG signal by integrating time-frequency domain and differential entropy features provided by the present invention is applicable to downstream tasks such as epilepsy detection, sleep staging and emotion recognition. A further detailed description is given.

[0057] A method for generating an electroencephalogram signal comprises the following steps:

[0058] Step (1) Dataset division:

[0059] The original EEG data set is divided into training data and test data. The original data set contains 152 subjects, 142 subjects are randomly selected as training samples, and the remaining 10 subjects are used as test samples. The EEG frequency in SLEEPEDFX is 100HZ. Each segment of subject data is split into multiple 30s data segments, and the FPZ channel data related to sleep is selected for experiments. Therefore, the dimension of each segment of input data is 1×3000, where 1 is the number of channels and 3000 is the time-frequency feature of 30×100.

[0060] Step (2) Time-frequency feature extraction: In order to obtain the local analysis capability of the signal in time and frequency, the original EEG signal is subjected to continuous wavelet transform. The transformation formula can be expressed as:

[0061]

[0062] Where x(t) is the input EEG signal, ψ * (t) is the complex conjugate of the wavelet function ψ(t), and the commonly used cgau8 is selected as the wavelet function. a is the scale parameter, which controls the expansion and contraction of the wavelet function. The parameter a is set to 8, which means that the total frequency band is divided into a-1, that is, 7 frequency bands. For each scale a, a series of wavelet coefficients are obtained, which vary with the translation parameter τ. τ is the translation parameter that controls the position of the wavelet function on the time axis. τ can be any real number, which means that the continuous wavelet transform allows non-integer unit translation analysis of the signal on the time axis. Repeating the above process, for different scales a, feature vectors at different frequency resolutions can be obtained. The original EEG of 1×3000 finally obtains a complex TF feature of 7×3000 dimensions.

[0063] Step (3) Differential entropy feature extraction: Each 30-second EEG signal is further decomposed into multiple 1-second data segments. Each 1-second data segment is divided into five frequency bands: δ wave (0.5-4 Hz), θ wave (4-8 Hz), α wave (8-13 Hz), β wave (13-32 Hz) and γ wave (32-50 Hz) according to the brain wave frequency range using a filter. For each frequency band, calculate its variance σ 2 The differential entropy calculation formula is:

[0064]

[0065] Where e represents the base of natural logarithm, and π refers to pi.

[0066] Finally, DE features with 1×150 dimensions are extracted.

[0067] Step (4) establishes and trains the EEG generation model; specifically:

[0068] 4.1 Construct a VAE dual encoder network and encode the time-frequency features and time-frequency features to obtain latent space representation;

[0069] 4.2 Construct a VAE decoder network; input standard Gaussian noise into the VAE decoder network to obtain generated EEG, and input the latent space representation into the VAE decoder network to obtain reconstructed EEG;

[0070] 4.3 Construct a discriminator with a residual network to identify the original EEG in the training set in step 1 and the EEG generated and reconstructed in step 5;

[0071] 4.4 Repeat the training process of steps 4.1-4.3 until the VAE decoder network and the discriminator with residual network reach a balance, and the VAE decoder network reaches the ability to expand the data set;

[0072] In step 4.1, construct a VAE dual encoder network to obtain latent space representation:

[0073] like Figure 2 The VAE dual encoder network includes a time-frequency encoder, a differential entropy encoder, and a latent space layer; the input of the time-frequency encoder is a TF feature with a dimension of 7×1×3000, and the input of the differential entropy encoder is a DE feature with a dimension of 1×150.

[0074] The time-frequency encoder includes three layers of complex convolutional neural networks and fully connected layers connected in series. The three-layer complex convolutional neural network is a three-layer convolutional layer, and the input of each layer is composed of real data Xr and imaginary data Xi of the same dimension; the activation functions of the three convolutional layers are LeakyRelu, and a batch normalization layer (BatchNormld) is added before the activation function to normalize the data. The time-frequency encoder receives input data of 7×1×3000 dimensions, and after passing through a complex two-dimensional convolutional layer with 7 input channels, a step size of 2, a convolution kernel of 3×3, and an output channel number of 8, the feature map is passed into the second layer of complex two-dimensional convolutional layer. The second layer of input data is input to the third layer of complex two-dimensional convolutional layer after passing through a complex two-dimensional convolutional layer with 8 input channels, a step size of 2, a convolution kernel of 3×3, and an output channel number of 16. The third complex convolution layer has 16 input channels, a stride of 2, a convolution kernel of 3×3, and 8 output channels. A 6000×70 fully connected layer is used to take the absolute value of the output of the third complex convolution layer and then reshape it to obtain the first latent space distribution with a dimension of 70.

[0075] The differential entropy encoder includes three layers of one-dimensional convolutional layers and fully connected layers connected in series. The fully connected layer is used to perform dimensionality reduction processing on the outputs of the three one-dimensional convolutional layers. The differential entropy encoder passes the 1×150-dimensional DE features through the first one-dimensional convolutional layer with an input channel number of 1, a step size of 2, a convolution kernel of 3, and an output channel number of 16, and then passes the feature map into the second one-dimensional convolutional layer. The second one-dimensional convolution has an input channel number of 16, a step size of 2, a convolution kernel of 3, and an output channel number of 32, and the output is passed to the third complex convolutional layer. The third complex convolutional layer has an input channel number of 32, a step size of 2, a convolution kernel of 3, and an output channel number of 32. In order to meet the input requirements of the fully connected layer, the output of the third layer is reshaped to obtain a 1248-dimensional extracted feature, and a second latent space distribution with a dimension of 30 is obtained through a 1248×30 fully connected layer.

[0076] Use the hidden space layer to distribute the first hidden space and the second latent space distribution Splicing, get the final latent space distribution with a dimension of 100 It is the random noise in latent space.

[0077] In step 4.2, step (5) constructs a VAE decoder network, i.e., a generator, to generate EEG signals;

[0078] The input of the generator is a randomly sampled noise with a dimension of 100. The structure includes the first deconvolution layer, the second deconvolution layer, the first self-attention layer and two convolution layers connected in series, as shown in Figure 2As shown in the decoder in . Before being placed in the first deconvolution layer, it is necessary to pass through a 100×2992-dimensional fully connected layer, and the output 2992-dimensional data is reshaped to obtain a 16×187×1 dimension to meet the input requirements of the two-dimensional deconvolution layer. The convolution kernel of the deconvolution layer is set to 4, that is, the unit of the convolution layer receives the input data dimension of 1×1 and reversely outputs 4×4 output data. The model of the present invention adopts a two-layer deconvolution structure, and finally the output of the second deconvolution layer is passed through a self-attention layer to learn the relationship between each channel. The data input to the self-attention layer is reshaped into 64×3000 dimensional data, and three identical matrices Q, K, and V are copied and generated. Through the one-dimensional convolution method, the shapes of Q and K are adjusted to 8×3000 matrices, and the V matrix is ​​maintained at 64×3000 for subsequent matrix multiplication. The product of Q and K is calculated using batch matrix multiplication to obtain an attention score matrix with a shape of 3000×3000, and the softmax function is applied to the attention score matrix to obtain a normalized attention weight matrix. V is further multiplied by the transpose of the attention weight matrix to obtain a weighted sum, the shape of which is the same as the original input. The weighted sum is scaled using the learnable attenuation parameter gamma, and the original input is added to the weighted sum, ultimately achieving mutual learning between the data of each channel. Finally, the 64×3000 dimensional data output by the attention layer is passed through a one-dimensional convolutional layer with 64 input channels and 1 output channel to obtain a generated 3000-dimensional EEG signal. The decoder inputs latent space noise to obtain reconstructed data, and inputs standard Gaussian noise to obtain generated data.

[0079] In step 4.3, a discriminator network is constructed to distinguish true and false samples:

[0080] The generated data in step 4.2 and the real data in step 1 are input into the discriminator to judge the authenticity of the input signal. The discriminator structure of this model is as follows Figure 2As shown in the discriminator, the structure includes the first convolution layer, the second convolution layer, the third convolution layer, the second self-attention layer and two fully connected layers connected in series. The input of the discriminator is 3000-dimensional original EEG data and generated EEG data. In order to meet the input requirements of the convolution layer, the data is reshaped into 1×3000-dimensional input. The discriminator uses a one-dimensional convolution layer with a convolution kernel size of 3, and the number of output channels of each convolution layer increases step by step to 16, 32, and 64. As the number of convolution layers increases, more advanced features are extracted. Each convolution layer performs residual processing, and the input features are added to the output features. After each convolution layer is processed by residual processing, the average pooling layer with an input kernel size of 2 is used to reduce the dimension of the features. After the convolution layer, there is a self-attention mechanism layer, which generates three matrices Q, K, and V from the input 64×300-dimensional data. Through the one-dimensional convolution method, the shapes of Q and K are adjusted to 8×300 matrices, and the V matrix is ​​maintained at 64×300 for subsequent matrix multiplication. The product of Q and K is calculated using batch matrix multiplication to obtain an attention score matrix with a shape of 300×300. The softmax function is applied to the attention score matrix to obtain a normalized attention weight matrix. V is further multiplied by the transpose of the attention weight matrix to obtain a weighted sum, and the shape of the weighted sum is the same as the original input. The weighted sum is scaled using a learnable attenuation parameter gamma, and the original input is added to the weighted sum. Finally, the output data is reshaped into 19200 dimensional data and mapped to 1-dimensional data through two fully connected layers of 19200×256 and 256×1 dimensions, representing the discriminator's discrimination result for the input data.

[0081] Compared with traditional EEG generation methods such as cropping, splicing, and noise addition, the method proposed in the present invention generates data by learning the true generation distribution of EEG signals, expanding the size of the EEG dataset, and providing sufficient datasets for deep learning models to complete training. This can improve the accuracy of downstream tasks and reduce the cost of EEG acquisition.

[0082] Step 4.4 During network training:

[0083] For the latent space mean and variance obtained by the encoder, the KL divergence is used to calculate the loss to encourage the latent variable distribution to be close to the prior distribution to improve the generalization ability of the latent space. The formula is:

[0084]

[0085] Where P represents the latent space distribution of dimension 100 Q represents the standard Gaussian distribution Q~N(0,1), and the following formula is obtained by simplifying it:

[0086]

[0087] μi and σ i is the i-th component of the mean and variance of the posterior distribution P. The VAE encoder network parameters θ1 are obtained by training.

[0088] For the reconstructed data output by the generated network with the self-attention mechanism, it is input into the discriminator to obtain the similarity loss The formula is:

[0089]

[0090] Where P represents the latent space distribution, z r represents random noise sampled from the latent space distribution, G is the decoder, and D is the discriminator.

[0091] The reconstructed data obtained by the decoder is further compared with the real data to obtain the reconstruction loss:

[0092]

[0093] Where MSE is the mean square loss function, P data(x) Represents the original data distribution. The decoder network parameters θ2 are obtained by training.

[0094] Therefore, the overall loss of the VAE composed of the encoder and decoder is:

[0095]

[0096] For the identification results obtained by the discriminator, the cross entropy loss function is used to calculate the network loss. The real data and generated data are input into the discriminator to determine the authenticity of the input signal to obtain the real identification loss L1 and the generated identification loss L2 and train the discriminator. The penalty loss L3 is also added to the parameters of the discriminator itself, and the discriminator loss is obtained as follows:

[0097]

[0098] Where z represents standard Gaussian noise, λ is an adjustable hyperparameter, representing the loss coefficient, which is used to control the size of the discriminator network parameter θ3. The initial value is 10. After every five trainings, λ becomes 0.9 times the original value. As the adversarial training proceeds, the adversarial relationship between the generator and the discriminator eventually reaches a balance, and the generator is able to expand the data set.

[0099] Step 5: Use the trained VAE decoder network to obtain the optimal generated EEG data, and test and verify the optimal generated EEG data and the original EEG data of the test set; finally, use the trained and tested VAE decoder network to realize EEG signal amplification.

[0100] In order to demonstrate the effectiveness of the method of the present invention in generating EEG data, the SLEEPEDFX dataset is used in combination with the commonly used generation indicator FID value to verify the effectiveness of the generated samples. The FID (Fréchet Inception Distance) generation indicator is an indicator used to measure the gap between two image distributions. It is widely used in the evaluation of image generation models such as generative adversarial networks (GANs) and variational autoencoders (VAEs). FID evaluates the similarity between the generated image and the real image by calculating the mean and covariance matrix of the two distributions and calculating the Fréchet distance between them. The FID indicator formula is as follows:

[0101] FID(P t ,G)=||μ p -μ g ||2+Tr(σ p +σ g -2(σ p σ g )1 / 2)

[0102] in and Represents the distribution of real test images and the distribution of generated images respectively. p and μ g Represent the mean vectors of P and G respectively. σ p and σ g Respectively represent the covariance matrices of P and G. Tr represents the trace of the matrix, that is, the sum of the diagonal elements of the matrix.

[0103] The random noise sampled from the standard Gaussian distribution is put into the decoder using the optimized estimated network parameter θ2 to obtain EEG data generated by random noise. At the same time, an EEG generation model is constructed, i.e., the VAE-GAN model and the WGAN model are used as the basic model for comparison with the model of the present invention, and the EEG data are generated by training using the same parameters. The FID value of the data generated by the above three models is calculated simultaneously with the test data divided in step (1). The smaller the FID value, the more similar the generated sample distribution is.

[0104] The following Table 1 shows the FID values ​​calculated for data generated by different models and original data. It can be seen intuitively that the model of the present invention has a higher similarity than other models and original data.

[0105] Table 1

[0106] Using the Model Model of the present invention WGAN VAEGAN FID value 8.1 11.46 69

[0107] In order to further demonstrate the robustness of the model of the present invention, an additional robustness test is conducted, that is, the generated data is only used to calculate the FID value with the test data of a single subject. The calculated FID values ​​are shown in Table 2.

[0108] Table 2

[0109] Using the Model Model of the present invention WGAN VAEGAN FID value 11.39 38.84 94

[0110] When the generation index test experiment was conducted on a single subject, it was found that other models all had a similarity landslide phenomenon, while the model of the present invention was only affected to a limited extent, which further proved that this experiment can generate more universal samples.

[0111] This experimental example also observed the EEG topography of the data generated and the original data of the SLEEPEDFX dataset, further supporting the previous conclusion. Figure 3A-3B The EEG topography map drawn by the mean of all real samples and generated samples in the same frequency band is shown. By observing the image, it can be seen that the difference between the generated EEG features and the real EEG features is not obvious. Compared with the real EEG signal, the generated EEG data can be well matched in terms of the relationship between channels and the data size on the channels, and the details and rules of the real EEG signal can be learned. This shows that the model of the present invention has excellent ability in learning the characteristics of EEG data and can well maintain the morphological characteristics of the real data.

[0112] Figure 4 The T-SNE dimension reduction algorithm diagram of the generated data and real data generated by the model of the present invention for a subject in another public dataset SEED emotion dataset is shown. The dot data is the generated data, and the bar data is the real data. It can be seen from the figure that the corresponding generated data gathers with the real data of the same emotion, and separates the three emotions without overlap. The reliability and effectiveness of the model of the present invention for learning different features are verified. Figure 4 It also shows that the generated data is at the center of classification decisions, which helps to clarify classification boundaries when performing downstream classification tasks, thereby improving classification accuracy, further demonstrating the superiority of the model of the present invention.

Claims

1. A method for generating EEG signals by integrating time-frequency domain and differential entropy features, characterized in that The steps include: Step 1: Divide the original EEG data into training set and test set in proportion; Step 2: For the original EEG signals in the training set, calculate and obtain the time-frequency characteristics of the EEG signals; Step 3: For the original EEG signals in the training set, calculate and obtain the differential entropy features of the EEG signals; Step 4: Establish and train the EEG generation model; specifically: 4.1 Construct a VAE dual encoder network and encode it according to the time-frequency features and differential entropy features to obtain the latent space representation; 4.2 Construct a VAE decoder network; input standard Gaussian noise into the VAE decoder network to obtain generated EEG, and input the latent space representation into the VAE decoder network to obtain reconstructed EEG; 4.3 Construct a discriminator with a residual network to identify the original EEG in the training set in step 1 and the EEG generated and reconstructed in step 5; 4.4 Repeat the training process of steps 4.1-4.3 until the VAE decoder network and the discriminator with residual network reach a balance, and the VAE decoder network reaches the ability to expand the data set; Step 5: Use the trained VAE decoder network to obtain the optimal generated EEG data, and test and verify the optimal generated EEG data and the original EEG data of the test set; finally, use the trained and tested VAE decoder network to realize EEG signal amplification.

2. The method for generating an EEG signal integrating time-frequency domain and differential entropy features according to claim 1, characterized in that: The VAE dual encoder network described in step 4.1 includes a time-frequency encoder, a differential entropy encoder, and a latent space layer; the time-frequency encoder is used to calculate the first latent space distribution of the time-frequency features. The differential entropy encoder is used to calculate the second latent space distribution of the differential entropy feature. Finally, the hidden space layer is used to distribute the first hidden space and the second latent space distribution Splicing to get the final latent space distribution It is the random noise in latent space.

3. The method for generating an EEG signal integrating time-frequency domain and differential entropy features according to claim 2, characterized in that: The time-frequency encoder includes three layers of complex convolutional neural networks and a fully connected layer connected in series.

4. The method for generating an EEG signal integrating time-frequency domain and differential entropy features according to claim 2, characterized in that: The differential entropy encoder includes three one-dimensional convolutional layers and a fully connected layer connected in series.

5. The method for generating an EEG signal integrating time-frequency domain and differential entropy features according to claim 1, characterized in that: The VAE decoder network described in step 4.2 includes a first deconvolution layer, a second deconvolution layer, a first self-attention layer, and two convolution layers connected in series.

6. The method for generating an EEG signal integrating time-frequency domain and differential entropy features according to claim 1, characterized in that: The discriminator with residual network described in step 4.3 includes a first convolutional layer, a second convolutional layer, a third convolutional layer, a second self-attention layer and two fully connected layers connected in series in sequence; a residual connection is also provided between the first convolutional layer and the second convolutional layer, and a residual connection is also provided between the second convolutional layer and the third convolutional layer; the first convolutional layer, the second convolutional layer, and the third convolutional layer have the same structure, and all include a one-dimensional convolutional layer and an average pooling layer connected in series.

7. The method for generating an EEG signal integrating time-frequency domain and differential entropy features according to claim 6, characterized in that: The loss function of the EEG generation model includes the loss function L of the VAE dual encoder network and the VAE decoder network VAE , the loss function L of the discriminator with residual network D ; The loss functions of the VAE dual encoder network and the VAE decoder network are as follows: KL divergence loss: Where P represents the latent space distribution Q represents the standard Gaussian distribution Q~N(0,1), M represents the dimension of the latent space, μ i and σ i is the i-th component of the mean and variance of the posterior distribution P; Similarity loss: Where P represents the latent space distribution, z r represents random noise sampled from the latent space distribution, G is the VAE decoder network, and D is the discriminator; Represents sampling noise z from the latent space r Expected after decoder and discriminator; reconstruction loss: Where MSE is the mean square loss function, P data(x) Represents the original data distribution; Represents the original EEG x and the reconstructed EEG G(z r ) is the expected mean square error between The loss function of the discriminator with residual network is as follows: Where z represents standard Gaussian noise, λ represents an adjustable hyperparameter, θ3 represents the discriminator network parameter; N(0,1) represents the standard normal distribution; ∥·∥2 represents the L2 norm.

8. An electroencephalogram signal generation system based on the method according to any one of claims 1 to 7, characterized in that include: Data acquisition module, used to extract the time-frequency characteristics and differential entropy characteristics of the original EEG signal; The EEG generation module inputs the time-frequency characteristics and differential entropy characteristics of the original EEG signal into the EEG generation model to realize EEG signal amplification.

9. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the method according to any one of claims 1 to 7.

10. A machine-readable storage medium, characterized in that: The machine-readable storage medium stores machine-executable instructions. When the machine-executable instructions are called and executed by a processor, the machine-executable instructions prompt the processor to implement the method described in any one of claims 1 to 7.

Images