Electroencephalogram signal anxiety emotion assessment method based on unbalanced data driving

Abstract

The invention provides an electroencephalogram signal anxiety emotion evaluation method based on unbalanced data driving, and relates to the technical field of anxiety emotion evaluation.The method comprises the steps that an anxiety electroencephalogram data set is obtained, preprocessed and divided into a training set and a test set; constructing a data enhancement model, and inputting the training set data into the data enhancement model for training; providing noise vectors randomly sampled from standard normal distribution and distributed labels for the trained data enhancement model, generating a least class of samples, and combining the training set and the least class of samples to obtain a balanced anxiety electroencephalogram training set; constructing an anxiety emotion evaluation model, and training the anxiety emotion evaluation model by using the balanced anxiety electroencephalogram data; and processing the electroencephalogram test set by using the trained anxiety emotion evaluation model to obtain an anxiety emotion evaluation result. According to the method, a certain type of signals can be generated in a targeted mode to obtain the balanced electroencephalogram data set, and the accuracy of electroencephalogram signal anxiety emotion evaluation is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of anxiety emotion assessment, and in particular relates to an anxiety emotion assessment method based on electroencephalogram (EEG) signals driven by unbalanced data. Background Art

[0002] Anxiety is a common emotional response. Being in a state of anxiety for a long time will have a serious impact on people's daily life and work. At present, the detection of anxiety and the diagnosis of anxiety disorders usually rely on questionnaires and clinical assessments, which are relatively subjective. Anxiety assessment based on EEG signals realizes a non-invasive and objective assessment method by analyzing specific neural activity patterns or characteristics.

[0003] In the field of EEG anxiety recognition, the commonly used method is to pre-extract the differential entropy features or power spectral density features of the EEG signal and input it into the network for classification, or to convert the signal into a time-frequency image and input it into the network for classification. However, the original EEG signal is random and contains more emotional feature information, so some studies choose to directly input the original EEG signal into the neural network to retain more details and dynamic information. But at the same time, this method may also introduce noise and unnecessary complexity, and may require more complex algorithms and model designs to effectively extract useful information.

[0004] In real life, anxiety EEG data has imbalanced distribution of samples and categories, while existing deep learning algorithms are generally designed based on roughly balanced distribution of sample categories. Training anxiety assessment-related models on unbalanced datasets may cause the model to favor the majority category and ignore the minority category, thus affecting the accuracy and robustness of the assessment method. In response to this problem, relevant research in the field of EEG signal classification is still insufficient. With the development of deep learning technology, some studies have begun to explore the application of autoencoders and generative adversarial networks in EEG signal generation, but this deep generative model has problems such as mode collapse, unstable training, or single generated samples.

[0005] Currently, the use of EEG signals for anxiety assessment still faces challenges such as data imbalance and insufficient model generalization. In addition, most existing studies are limited to a single binary classification task, and there are relatively few studies on multi-classification of anxiety levels. Summary of the invention

[0006] In view of the above-mentioned deficiencies in the prior art, the present invention provides an EEG signal anxiety emotion assessment method based on imbalanced data driven, which introduces variational autoencoders and channel attention mechanisms in conditional generative adversarial networks to improve the quality of generated signals and solve the existing problem of data imbalance in EEG signal anxiety emotion assessment; Transformer networks and convolutional neural networks are selected and improved to improve the model's ability to extract relevant EEG features; the present invention effectively improves the accuracy and generalization ability of the EEG signal anxiety emotion assessment method by data expansion, improving network structure, etc.

[0007] In order to achieve the above purpose, the technical solution adopted by the present invention is: a method for evaluating anxiety emotion based on EEG signals driven by imbalanced data, comprising the following steps:

[0008] S1, obtaining an anxiety EEG data set, preprocessing the obtained anxiety EEG data set, and dividing the preprocessed anxiety EEG data set into a training set and a test set;

[0009] S2. Build a data enhancement model based on variational autoencoder and conditional generative adversarial network, and input the training set data into the data enhancement model for training;

[0010] S3, providing the trained data augmentation model with a noise vector randomly sampled from a standard normal distribution and an assigned label to generate a minimum class sample, and combining the training set with the minimum class sample to obtain a balanced anxiety EEG training set;

[0011] S4. Construct an anxiety emotion assessment model based on convolutional neural network and Transformer, and use balanced anxiety EEG data to train the anxiety emotion assessment model;

[0012] S5. Use the trained anxiety emotion assessment model to process the EEG test set to obtain the anxiety emotion assessment result, and complete the anxiety emotion assessment of the EEG signal.

[0013] Furthermore, the step S1 includes the following steps:

[0014] S101, obtaining anxiety EEG data set;

[0015] S102, using a filter to remove noise from the anxiety EEG dataset;

[0016] S103, perform independent component analysis on the anxiety EEG dataset after noise removal;

[0017] S104, based on the anxiety EEG data set after independent component analysis, arousal and valence scores were marked to obtain four categories: normal, mild anxiety, moderate anxiety, and severe anxiety;

[0018] S105, segmenting the EEG signal according to the labeled anxiety EEG data set;

[0019] S106, mapping the segmented anxiety EEG data set to [-1, 1] using maximum and minimum normalization to complete the preprocessing of the anxiety EEG data set;

[0020] S107, dividing the preprocessed anxiety EEG data set into a training set and a test set.

[0021] Further, the data enhancement model includes two adversarial learnings; the first adversarial learning includes a first encoder module and an encoding discriminator module; the second adversarial learning includes a second encoder module, a generator module and a generating discriminator module;

[0022] The first encoder module is used to obtain training set samples and learn the distribution of the input training set samples in the latent space to obtain the latent vectors of the training samples in the latent space;

[0023] The encoding discriminator module is used to obtain the latent vector z enc and a noise vector z randomly sampled from a standard normal distribution rand ;

[0024] The generator module is used to generate a latent vector z according to the real sample enc , noise vector z rand And the corresponding category labels, output reconstructed samples and newly generated samples;

[0025] The second encoder module is used to embed the label information into the latent vector z enc In a vector of the same size, combine the label information and the latent vector z enc Get a labeled latent vector and decode it back to its original dimension;

[0026] The generated discriminator module is used to obtain the total validity score of true and false and class matching according to the sample pair, wherein the sample pair includes the real sample x real and the true sample label y real Sample pair, reconstructed sample x enc and reconstruct the sample label y enc Sample pairs and newly generated samples x rand And generate sample labels y rand Sample pairs.

[0027] Furthermore, the first encoder module and the second encoder module have the same structure, both comprising: three first temporal convolutional layer branches, a first channel attention, a first spatial convolutional layer connected to the first channel attention, a second channel attention connected to the first spatial convolutional layer, a first depthwise separable layer connected to the second channel attention, a first flattening layer connected to the first depthwise separable layer, and a first fully connected layer connected to the first flattening layer;

[0028] The encoding discriminator module includes a plurality of second fully connected layers connected in sequence;

[0029] The generator module includes a first label encoding layer, a third fully connected layer connected to the label encoding layer, a reshaping layer connected to the third fully connected layer, a fourth fully connected layer connected to the reshaping layer, a transposed convolution layer connected to the fourth fully connected layer, and a self-attention layer connected to the transposed convolution layer;

[0030] The generative discriminator module includes three second temporal convolutional layers, a third channel attention, a second spatial convolutional layer connected to the third channel attention, a fourth channel attention connected to the second spatial convolutional layer, a second flattening layer connected to the fourth channel attention, and a fifth fully connected layer connected to the second flattening layer and the second label encoding layer.

[0031] Furthermore, the training process of the data enhancement model is as follows:

[0032] A1. Obtain the imbalanced anxiety EEG training set;

[0033] A2. In the first part of each training epoch, the training set samples are input into the first encoder module to learn the distribution of the input training set samples in the latent space and obtain the latent vector z of the training samples in the latent space. enc ;

[0034] A3. The latent vector z enc and a noise vector z randomly sampled from a standard normal distribution rand Input to the encoding discriminator module to determine the authenticity of the input data;

[0035] A4. According to the judgment result, train the first encoder module and the encoder discriminator module, and update the parameters of the first encoder module and the encoder discriminator module;

[0036] A5. In the second part of each training epoch, the generator module receives the latent vector z of the real sample. enc , noise vector z rand And the corresponding category label, output the reconstructed sample and the newly generated sample; wherein the second encoder module embeds the label information into the latent vector z encIn a vector of the same size, combine the label information and the latent vector z enc Get a labeled latent vector and decode it back to its original dimension;

[0037] A6. Input the sample pair into the generative discriminator module to obtain the total validity score of true and false and class matching, where the sample pair includes the real sample x real and the true sample label y real Sample pair, reconstructed sample x enc and reconstruct the sample label y enc Sample pairs and newly generated samples x rand And generate sample labels y rand Sample pairs;

[0038] A7. Use the total validity score of true and false and class matching to train the second encoder module, generator module and generated discriminator, and update the parameters of the second encoder module, generator module and generated discriminator to complete the training of the data enhancement model.

[0039] Furthermore, the loss function of the first adversarial learning is expressed as follows:

[0040]

[0041] E(x real )=z enc

[0042] Where L1 represents the loss function of the first adversarial learning, E(·) represents the operation of the first encoder module, and D E (·) represents the operation of the encoding discriminator module, Represents a given latent vector E(x real ) obeys the probability distribution P e Under the condition of real ), E(x real ) represents the potential vector of the real data output by the encoder, P e Represents the potential vector E(x) output by the encoder of the real data real ) obeys the probability distribution, Represents a given latent vector z rand Obey the probability distribution P t Under the condition of rand The conditional expectation of z rand represents a noise vector randomly sampled from a standard normal distribution, P t represents the noise vector z rand The probability distribution is obeyed, represents the weight of the gradient penalty, represents the gradient calculation, L GPrepresents the loss of the gradient penalty term, Indicates from z enc and z rand The data sampled from the mixed sample, represents the sampled latent vector, express The probability distribution that follows is E(x real ) and z enc Both represent latent vectors;

[0043] The expression of the loss function of the second adversarial learning is as follows:

[0044] L2=L real,enc +L real,rand

[0045]

[0046] G(z enc )=x enc

[0047]

[0048] Among them, L2 represents the loss function of the second adversarial learning, L real,enc Represents the real sample x real and reconstruct the sample x enc The distance metric between real,rand Represents the real sample x real and the newly generated sample x rand The distance measure between Represents the reconstructed sample G(z enc ) obeys the probability distribution P ze Under the condition of enc ), G(·) represents the operation of the generator, D G (·) represents the operation of generating the discriminator, G(z enc ) and x enc All represent reconstructed samples. Represents the real sample x real Obey the probability distribution P r Under the condition of real The conditional expectation of L GP-1 Represents the real sample x real and reconstruct the sample x enc The loss of the gradient penalty term during adversarial training corresponds to the loss of the gradient penalty term L GP , represents the reconstruction loss, λ1 represents the weight of the reconstruction loss, Represents the newly generated sample x rand Obey the probability distribution P zrUnder the condition of rand The conditional expectation of G(z rand ) and x rand Both represent newly generated samples, L GP-2 Represents the real sample x real and the newly generated sample x rand Loss of the gradient penalty term during adversarial training.

[0049] Furthermore, the anxiety emotion assessment model in step S5 includes a spatiotemporal Transformer module, a dual-branch convolution module and a classification module connected in sequence;

[0050] The spatiotemporal Transformer module includes a position encoding submodule, a time Transformer submodule and a space Transformer submodule both connected to the position encoding submodule, and a feature fusion submodule both connected to the time Transformer submodule and the space Transformer submodule;

[0051] The dual-branch convolution module includes a temporal convolution submodule, a fifth channel attention, a deep convolution submodule connected to the fifth channel attention, and a sixth channel attention connected to the deep convolution submodule; wherein the temporal convolution submodule includes three third temporal convolution layers with different kernel sizes, and the deep convolution submodules each include two branches, each branch including a spatial convolution layer for obtaining multi-scale spatial features from a feature map and a spatiotemporal convolution layer for learning to fuse multi-scale spatiotemporal features;

[0052] The classification module includes a third flattening layer, a sixth fully connected layer and a softmax layer which are connected in sequence.

[0053] Furthermore, the position encoding submodule is used to add position information to the EEG signal;

[0054] The temporal Transformer submodule is used to extract the correlation between different time points of the multi-channel EEG signals according to the EEG signals with added position information to obtain the temporal features;

[0055] The spatial Transformer submodule is used to extract the correlation between different channels of the multi-channel EEG signal according to the EEG signal with added position information to obtain the spatial feature;

[0056] The feature fusion submodule is used to fuse the temporal features and the spatial features to obtain a feature vector F;

[0057] The time convolution submodule is used to obtain the multi-scale time features of the multi-channel EEG signal according to the feature vector F;

[0058] The deep convolution submodule is used to extract the spatiotemporal features of the EEG signal according to the multi-scale time features, and fuse the multi-scale time features with the spatiotemporal features by using a summation operation to obtain the deep spatiotemporal features of the EEG signal;

[0059] The third flattening layer is used to convert the multi-dimensional deep spatiotemporal features output by the dual-branch convolution module into one-dimensional features;

[0060] The sixth fully connected layer is used to perform linear transformation on the deep spatiotemporal features of the EEG signal to obtain the prediction probability;

[0061] The softmax layer is used to output a predicted classification result according to the predicted probability, wherein the classification results include normal, mild anxiety, moderate anxiety and severe anxiety.

[0062] Furthermore, the training process of the anxiety assessment model is as follows:

[0063] B1. Obtain the balance anxiety EEG training set;

[0064] B2. Input the balance anxiety EEG training set into the spatiotemporal Transformer module and fuse it to obtain the feature vector F, in which the EEG signals are arranged in both space and time.

[0065] B3, using the fused feature vector F, using a double-branch convolution module to obtain the deep spatiotemporal features of the EEG signal;

[0066] B4. According to the deep spatiotemporal characteristics of the EEG signal, the classification module is used to obtain the prediction probability, and the prediction classification result is obtained according to the prediction probability;

[0067] B5. Use the predicted classification results and the true classification labels to calculate the value of the loss function of the anxiety emotion assessment model, and back-propagate the gradient according to the value of the loss function to update the parameters of the anxiety emotion assessment model until convergence, thus completing the training of the anxiety emotion assessment model.

[0068] Furthermore, the loss function of the anxiety assessment model is expressed as follows:

[0069]

[0070] Among them, L(x,y) represents the loss function of the anxiety assessment model, N represents the batch size, i represents the i-th sample, and l i represents the loss of the i-th sample, y represents the true category label, j represents the j-th category, C represents the number of categories, and x i,j It represents the probability that the i-th sample belongs to the j-th category.

[0071] Beneficial effects of the present invention:

[0072] (1) The present invention introduces channel attention in the conditional generative adversarial network to adjust the weights of different channels, and adds a coding discriminator to solve the error problem between the actual data distribution and the assumed distribution in the variational autoencoder, so as to improve the quality of generated samples.

[0073] (2) The present invention proposes to evaluate anxiety based on spatiotemporal Transformer and convolutional neural network, effectively extract useful information from multi-channel EEG signals, and can accurately identify the degree of anxiety, which has a positive impact on the early detection and intervention treatment of anxiety disorders.

[0074] (3) The present invention can solve the problem of unbalanced distribution of anxiety EEG data samples and categories in real life by generating a certain type of signal through a data enhancement model, and then accurately evaluate anxiety using an anxiety emotion assessment model. This unbalanced data-driven approach can better handle the unbalanced data distribution commonly seen in actual clinical practice and research, and improve the accuracy of the anxiety emotion assessment model. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 The figure is a flow chart of the method of the present invention.

[0076] Figure 2 It is a flowchart of the present invention.

[0077] Figure 3 Schematic diagram of the data enhancement model structure in this embodiment.

[0078] Figure 4 Schematic diagram of the anxiety emotion assessment model structure in this embodiment. DETAILED DESCRIPTION

[0079] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0080] Example

[0081] like Figure 1-Figure 2As shown, the present invention provides an EEG signal anxiety assessment method based on imbalanced data driven. The method generates minority class samples through a data enhancement model based on a variational autoencoder and a conditional generative adversarial network. The generated EEG signal retains the complete low-frequency and high-frequency features of the real EEG signal; and the anxiety assessment of patients with unknown labels is completed through an improved Transformer network and a convolutional neural network. The implementation method is as follows:

[0082] S1. Obtain an anxiety EEG data set, preprocess the obtained anxiety EEG data set, and divide the preprocessed anxiety EEG data set into a training set and a test set. The implementation method is as follows:

[0083] S101, obtaining anxiety EEG data set;

[0084] S102, using a filter to remove noise from the anxiety EEG dataset;

[0085] S103, perform independent component analysis on the anxiety EEG dataset after noise removal;

[0086] S104, based on the anxiety EEG data set after independent component analysis, arousal and valence scores were marked to obtain four categories: normal, mild anxiety, moderate anxiety, and severe anxiety;

[0087] S105, segmenting the EEG signal according to the labeled anxiety EEG data set;

[0088] S106, mapping the segmented anxiety EEG data set to [-1, 1] using maximum and minimum normalization to complete the preprocessing of the anxiety EEG data set;

[0089] S107, dividing the preprocessed anxiety EEG data set into a training set and a test set.

[0090] In this embodiment, the public dataset DASPS is used as the original input data. The public dataset DASPS is an anxiety EEG dataset. The experimental subjects include 13 women and 10 men. The EEG signals on 14 electrodes are recorded, namely F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8, AF4. The participants were asked to use the self-assessment model (SAM) to evaluate their feelings during the stimulation to express the arousal and valence emotions during the stimulation.

[0091] In this embodiment, the EEG data is preprocessed, including filtering, artifact removal, labeling, signal segmentation, normalization, and other operations, and then the processed EEG data is divided into a training set and a test set, as follows:

[0092] Use filters to remove unnecessary frequency components or noise; perform independent component analysis on the filtered EEG signals to remove artifacts such as electrooculography, electrocardiography, and electromyography; label all trials according to arousal and valence scores. Through this step, the DASPS anxiety EEG dataset can be divided into four categories, namely normal, mild anxiety, moderate anxiety, and severe anxiety, but the anxiety EEG dataset has a data imbalance problem; divide the EEG signals into 1-second segments to increase the original sample size; use the maximum-minimum normalization technique to map the EEG signal data to [-1, 1], and divide the preprocessed EEG signals into training and test sets according to the ratio of 8:2.

[0093] S2. Build a data enhancement model based on variational autoencoder and conditional generative adversarial network, and input the training set data into the data enhancement model for training;

[0094] S3, providing the trained data augmentation model with a noise vector randomly sampled from a standard normal distribution and an assigned label to generate a minimum class sample, and combining the training set with the minimum class sample to obtain a balanced anxiety EEG training set;

[0095] like Figure 3 As shown, the data enhancement model includes two adversarial learnings; the first adversarial learning includes a first encoder module and an encoding discriminator module; the second adversarial learning includes a second encoder module, a generator module, and a generating discriminator module;

[0096] The first encoder module is used to obtain training set samples and learn the distribution of the input training set samples in the latent space to obtain the latent vectors of the training samples in the latent space;

[0097] Encoder discriminator module to obtain the latent vector z enc and a noise vector z randomly sampled from a standard normal distribution rand ;

[0098] The generator module is used to generate the latent vector z according to the real sample enc , noise vector z rand And the corresponding category labels, output reconstructed samples and newly generated samples;

[0099] The second encoder module is used to embed the label information into the latent vector z enc In a vector of the same size, combine the label information and the latent vector z enc Get a labeled latent vector and decode it back to its original dimension;

[0100] Generate a discriminator module to obtain the total validity score of true and false and class matching based on the sample pair, where the sample pair includes the real sample x real and the true sample label yreal Sample pair, reconstructed sample x enc and reconstruct the sample label y enc Sample pairs and newly generated samples x rand And generate sample labels y rand Sample pairs.

[0101] In this embodiment, Figure 3 As shown, the first encoder module and the second encoder module have the same structure, both including: three first temporal convolutional layer branches, a first channel attention, a first spatial convolutional layer connected to the first channel attention, a second channel attention connected to the first spatial convolutional layer, a first depthwise separable layer connected to the second channel attention, a first flattening layer connected to the first depthwise separable layer, and a first fully connected layer connected to the first flattening layer;

[0102] The encoding discriminator module includes a plurality of second fully connected layers connected in sequence;

[0103] The generator module includes a first label encoding layer, a third fully connected layer connected to the label encoding layer, a reshaping layer connected to the third fully connected layer, a fourth fully connected layer connected to the reshaping layer, a transposed convolution layer connected to the fourth fully connected layer, and a self-attention layer connected to the transposed convolution layer;

[0104] The generative discriminator module includes three second temporal convolutional layers, a third channel attention, a second spatial convolutional layer connected to the third channel attention, a fourth channel attention connected to the second spatial convolutional layer, a second flatten layer connected to the fourth channel attention, and a fifth fully connected layer connected to the second flatten layer and the second label encoding layer.

[0105] The generative discriminator module performs feature extraction through the second temporal convolution layer, the second spatial convolution layer, the fourth channel attention mechanism and the second depth-separable layer, combines the obtained feature map with the label encoding information through the label encoding layer into a new feature vector, and then outputs a total validity score representing the true or false and class matching through the fifth fully connected layer.

[0106] In this embodiment, the training process of the data enhancement model is as follows:

[0107] A1. Obtain the imbalanced anxiety EEG training set;

[0108] In this embodiment, according to the data format of (batch size, number of channels, signal length), the samples of the unbalanced anxiety EEG training set are (n, 14, 128).

[0109] A2. In the first part of each training epoch, the training set samples are input into the first encoder module to learn the distribution of the input training set samples in the latent space and obtain the latent vector z of the training samples in the latent space.enc , where the data format is (n, 100).

[0110] In this embodiment, for the encoder module, the training set samples (n, 14, 128) are used as input, and different time scale features are extracted through three first time convolution layer branches (each time convolution layer includes a two-dimensional convolution layer and a batch normalization layer), and the different channels of the data are weighted through the first channel attention mechanism to better utilize multi-channel information. After passing through the first spatial convolution layer (including a two-dimensional convolution layer, a batch normalization layer, an ELU activation layer, and a pooling layer), channel weighted assignment is performed through the second channel attention mechanism, and then the output result of the second channel attention mechanism is processed using the first depth separable layer (including a two-dimensional convolution layer, a batch normalization layer, a two-dimensional convolution layer, a batch normalization layer, an ELU activation layer, and a pooling layer), and then the output potential vector (n, 100) is mapped through the first fully connected layer.

[0111] A3. The latent vector z enc and a noise vector z randomly sampled from a standard normal distribution rand Input to the encoding discriminator module to determine the authenticity of the input data;

[0112] In this embodiment, for the encoding discriminator module, the latent vector z enc and the noise vector z rand As input, it is mapped through three layers of the second fully connected layer and finally outputs a validity score indicating true or false.

[0113] A4. According to the judgment result, train the first encoder module and the encoder discriminator module, and update the parameters of the first encoder module and the encoder discriminator module;

[0114] In this embodiment, the first encoder module and the encoding discriminator module continuously update their parameters through adversarial training. Since the loss function of the traditional generative adversarial network is prone to problems such as mode collapse, the Wasserstein distance is used as the loss function of adversarial learning, and the gradient penalty mechanism is introduced to solve the mode collapse problem. Assume that E(·) represents the operation of the encoder, D E (·) represents the operation of encoding the discriminator, and the loss function of the first adversarial learning is:

[0115]

[0116] E(x real )=z enc

[0117] Where L1 represents the loss function of the first adversarial learning, E(·) represents the operation of the first encoder module, and D E(·) represents the operation of the encoding discriminator module, Represents a given latent vector E(x real ) obeys the probability distribution P e Under the condition of real ), E(x real ) represents the potential vector of the real data output by the encoder, P e Represents the potential vector E(x) output by the encoder of the real data real ) obeys the probability distribution, Represents a given latent vector z rand Obey the probability distribution P t Under the condition of rand The conditional expectation of z rand represents a noise vector randomly sampled from a standard normal distribution, P t represents the noise vector z rand The probability distribution is obeyed, represents the weight of the gradient penalty, represents the gradient calculation, L GP represents the loss of the gradient penalty term, Indicates from z enc and z rand The data sampled from the mixed sample, represents the sampled latent vector, express The probability distribution that follows is E(x real ) and z enc Both represent potential vectors.

[0118] A5. In the second part of each training epoch, the generator module receives the latent vector z of the real sample. enc , noise vector z rand And the corresponding category label, output the reconstructed sample and the newly generated sample; wherein the second encoder module embeds the label information into the latent vector z enc In a vector of the same size, combine the label information and the latent vector z enc Get a labeled latent vector and decode it back to its original dimension;

[0119] In this embodiment, for the generator module, the potential vector z enc and label information as input, and the label information is embedded into the latent vector z through the label embedding layer enc variables of the same size, and then the latent vector z encThe latent variable (n, 100) is obtained by multiplying it with the label vector, and then it passes through the third fully connected layer and the reshaping layer, and then through multiple transposed convolution layers (including one-dimensional transposed convolution layer, batch normalization layer, LeakyRELU activation layer). Then, it passes through a self-attention mechanism to learn the intrinsic relationship and dependency in the sequence, and finally passes through a transposed convolution layer and uses the Tanh activation function to map the output to [-1, 1] to generate a normalized EEG signal.

[0120] A6. Input the sample pair into the generative discriminator module to obtain the total validity score of true and false and class matching, where the sample pair includes the real sample x real and the true sample label y real Sample pair, reconstructed sample x enc and reconstruct the sample label y enc Sample pairs and newly generated samples x rand And generate sample labels y rand Sample pairs;

[0121] In this embodiment, for the generative discriminator module, the structure is similar to that of the encoder. The input sample is subjected to feature extraction through three second temporal convolutional layer branches (each branch includes a two-dimensional convolutional layer and a batch normalization layer), a third channel attention mechanism, a second spatial convolutional layer (including a two-dimensional convolutional layer, a batch normalization layer, an ELU activation layer, and a pooling layer), a fourth channel attention mechanism, and a second depth-separable layer (including a two-dimensional convolutional layer, a batch normalization layer, a two-dimensional convolutional layer, a batch normalization layer, an ELU activation layer, and a pooling layer), and then a feature vector (n, 256) is obtained through the second flattening layer. Then, the label encoding layer embeds the label information into a vector of the same size as the feature vector, and then multiplies the feature vector output by the second flattening layer with the feature vector embedded in the label to form a new feature vector (n, 256), which is mapped through the fifth fully connected layer to output a total validity score for true and false and class matching.

[0122] A7. Use the total validity score of true and false and class matching to train the second encoder module, generator module and generated discriminator, and update the parameters of the second encoder module, generator module and generated discriminator to complete the training of the data enhancement model.

[0123] In this embodiment, for the generated discriminator module, the real sample x real is considered to be a true sample, reconstructing the sample x enc and the newly generated sample x rand It is considered as a fake sample. The generator module and the generated discriminator module also perform adversarial learning. Assume that G(·) represents the operation of the generator module, D G (·) represents the operation of generating the discriminator. The loss function of the second adversarial learning consists of two parts: xreal and x enc The distance metric L between real,enc and x real and x rand The distance metric L between real,rand :

[0124]

[0125] Among them, G(z enc )=x enc The generator module outputs the latent vector z of the encoder enc Decode the generated reconstructed samples, Represents the reconstructed sample G(z enc ) obeys the probability distribution P ze Under the condition of enc ), G(z enc ) and x enc All represent reconstructed samples. Represents the real sample x real Obey the probability distribution P r Under the condition of real The conditional expectation of L GP-1 Represents x real and x enc The loss of the gradient penalty term during adversarial training corresponds to L GP , is the reconstruction loss, used to measure x enc and x real The difference between them, λ1 is the weight of the reconstruction loss, which ensures that the reconstructed samples that depend on the latent vector output by the encoder are as consistent as possible with the real samples;

[0126]

[0127] in, Represents the newly generated sample x rand Obey the probability distribution P zr Under the condition of rand The conditional expectation of G(z rand ) and x rand Both represent newly generated samples, L GP-2 Represents x real and x rand Loss of the gradient penalty term during adversarial training.

[0128] Therefore, the loss function L2 of the second adversarial learning is:

[0129] L2=L real,enc +L real,rand

[0130] S4. Construct an anxiety emotion assessment model based on convolutional neural network and Transformer, and use balanced anxiety EEG data to train the anxiety emotion assessment model;

[0131] S5. Use the trained anxiety assessment model to process the EEG test set, obtain the anxiety assessment results, and complete the anxiety assessment of EEG signals.

[0132] In this embodiment, Figure 4 As shown, the anxiety emotion assessment model in step S5 includes a spatiotemporal Transformer module, a dual-branch convolution module, and a classification module connected in sequence;

[0133] The spatiotemporal Transformer module includes a position encoding submodule, a time Transformer submodule connected to the position encoding submodule, a space Transformer submodule, and a feature fusion submodule connected to the time Transformer submodule and the space Transformer submodule;

[0134] The dual-branch convolution module includes a temporal convolution submodule, a fifth channel attention, a deep convolution submodule connected to the fifth channel attention, and a sixth channel attention connected to the deep convolution submodule; wherein the temporal convolution submodule includes three third temporal convolution layers with different kernel sizes, and the deep convolution submodules each include two branches, each branch including a spatial convolution layer for obtaining multi-scale spatial features from a feature map and a spatiotemporal convolution layer for learning to fuse multi-scale spatiotemporal features;

[0135] The classification module includes the third flattening layer, the sixth fully connected layer and the softmax layer which are connected in sequence.

[0136] The position encoding submodule is used to add position information to the EEG signal;

[0137] The temporal Transformer submodule is used to extract the correlation between multi-channel EEG signals at different time points based on the EEG signals with added position information to obtain the temporal features;

[0138] The spatial transformer submodule is used to extract the correlation between different channels of multi-channel EEG signals based on the EEG signals with added position information to obtain spatial features;

[0139] The feature fusion submodule is used to fuse the temporal features and spatial features to obtain the feature vector F;

[0140] The time convolution submodule is used to obtain the multi-scale time features of the multi-channel EEG signal according to the feature vector F;

[0141] The deep convolution submodule is used to extract the spatiotemporal features of the EEG signal based on the multi-scale time features, and fuse the multi-scale time features with the spatiotemporal features by summing them to obtain the deep spatiotemporal features of the EEG signal;

[0142] The third flattening layer is used to convert the multi-dimensional deep spatiotemporal features output by the dual-branch convolution module into one-dimensional features for subsequent full connection layer processing;

[0143] The sixth fully connected layer is used to perform linear transformation on the deep spatiotemporal features of the EEG signal to obtain the prediction probability;

[0144] The softmax layer is used to output the predicted classification results based on the predicted probability, where the classification results include normal, mild anxiety, moderate anxiety, and severe anxiety.

[0145] In this embodiment, the spatiotemporal Transformer module is composed of a position encoding module, a temporal Transformer module, a spatial Transformer module and a feature fusion block; the spatial Transformer module extracts the correlation between different channels of the multi-channel EEG signal, and the temporal Transformer module extracts the correlation between different time points of the multi-channel EEG signal; the feature fusion module adopts a gated fusion mechanism instead of simple addition or splicing, and the gated unit receives the output of the temporal Transformer module and the spatial Transformer module as input, and is responsible for adjusting the contribution of each feature to the overall output; assuming that the features output by the temporal Transformer module and the spatial Transformer module are represented by F1 and F2, then the feature vector F output by the feature fusion module can be expressed as:

[0146] F=αF1+(1-α)F2

[0147] α=sigmoid(W a [F1,F2])

[0148] Among them, W a represents the learnable weight parameter, and α represents the weight coefficient of the gating unit.

[0149] In this embodiment, the dual-branch convolution module is composed of a temporal convolution submodule, a deep convolution submodule, a fifth channel attention and a sixth channel attention; wherein, the temporal convolution module is mainly composed of three third temporal convolution layers with different kernel sizes, and the deep convolution module contains two branches, each branch includes a spatial convolution layer for obtaining spatial features from a feature map and a spatiotemporal convolution layer for learning how to correctly fuse spatiotemporal features, and both the temporal convolution submodule and the deep convolution submodule introduce a channel attention mechanism to enhance the learning ability of spatial information of EEG signals; the classification module includes a third flattening layer, a sixth fully connected layer and a softmax layer.

[0150] In this embodiment, the training process of the anxiety assessment model is as follows:

[0151] B1. Obtain the balance anxiety EEG training set;

[0152] In this embodiment, according to the data format of (batch size, number of channels, signal length), the EEG training samples are (n, 14, 128).

[0153] B2. Input the balance anxiety EEG training set into the spatiotemporal Transformer module and fuse it to obtain the feature vector F, in which the EEG signals are arranged in both space and time.

[0154] In this embodiment, the balance anxiety EEG training set is input into the spatiotemporal Transformer module, and the EEG signals are arranged in two ways: space (n, 14, 128) and time (n, 128, 14). The position information is added through the position encoding module, and then input into the spatial Transformer submodule and the temporal Transformer submodule to extract spatial features and temporal features, and the fused feature vector (n, 14, 128) is obtained after being processed by the feature fusion block.

[0155] B3, using the fused feature vector F, using a double-branch convolution module to obtain the deep spatiotemporal features of the EEG signal;

[0156] In this embodiment, the fused feature vector is input into the dual-branch convolution module, and the multi-scale temporal features of the multi-channel EEG signal are obtained through the time convolution submodule and the fifth channel attention processing. The output end of the fifth channel attention is connected to the input end of the deep convolution submodule to extract the spatiotemporal features of the EEG signal, and then the features are fused through the summation operation. At the same time, the sixth channel attention is introduced to enhance the learning ability of the spatial information of the EEG signal, and finally the deep spatiotemporal features of the EEG signal (n, 16, 1, 8) are obtained.

[0157] B4. According to the deep spatiotemporal characteristics of the EEG signal, the classification module is used to obtain the prediction probability, and the prediction classification result is obtained according to the prediction probability;

[0158] In this embodiment, the deep spatiotemporal features output by the dual-branch convolution module are input into the classification module, the output of the dual-branch convolution module is converted into a one-dimensional feature vector through the third flattening layer, and then the feature vector is linearly transformed through the sixth fully connected layer to obtain the predicted probability, and then the predicted classification result is output through the softmax layer, which is classified into four categories, namely normal, mild anxiety, moderate anxiety and severe anxiety.

[0159] B5. Use the predicted classification results and the true classification labels to calculate the value of the loss function of the anxiety emotion assessment model, and back-propagate the gradient according to the value of the loss function to update the parameters of the anxiety emotion assessment model until convergence, thus completing the training of the anxiety emotion assessment model.

[0160] In this embodiment, the anxiety assessment model uses the cross entropy loss function to calculate the model loss:

[0161]

[0162] Among them, L(x,y) represents the loss function of the anxiety assessment model, N represents the batch size, i represents the i-th sample, and l i represents the loss of the i-th sample, y represents the true category label, j represents the j-th category, C represents the number of categories, and x i,j It represents the probability that the i-th sample belongs to the j-th category.

[0163] In summary, by improving the conditional generative adversarial network, the minority class samples in the anxiety EEG dataset are effectively expanded, and by introducing the variational autoencoder and the channel attention mechanism, the quality of the generated signal is significantly improved, thereby solving the existing problem of data imbalance in EEG signal anxiety emotion assessment; the Transformer network and the convolutional neural network are selected and improved to enhance the model's ability to extract relevant EEG features, thereby improving the model's accuracy and generalization ability; the present invention can specifically generate a certain type of signal to obtain a balanced EEG dataset, and effectively improve the accuracy of the EEG signal anxiety emotion assessment method by expanding the minority class samples and improving the network structure.

Claims

1. A method for assessing anxiety based on EEG signals driven by imbalanced data, characterized in that: The following steps are involved: S1, obtaining an anxiety EEG data set, preprocessing the obtained anxiety EEG data set, and dividing the preprocessed anxiety EEG data set into a training set and a test set; S2. Build a data enhancement model based on variational autoencoder and conditional generative adversarial network, and input the training set data into the data enhancement model for training; S3, providing the trained data augmentation model with noise vectors randomly sampled from a standard normal distribution and assigned labels to generate minimum class samples, and combining the training set with the minimum class samples to obtain a balanced anxiety EEG training set; S4. Construct an anxiety emotion assessment model based on convolutional neural network and Transformer, and use balanced anxiety EEG data to train the anxiety emotion assessment model; S5. Use the trained anxiety emotion assessment model to process the EEG test set to obtain the anxiety emotion assessment result, and complete the anxiety emotion assessment of the EEG signal.

2. The method for evaluating anxiety based on EEG signals driven by imbalanced data according to claim 1, characterized in that: The step S1 comprises the following steps: S101, obtaining anxiety EEG data set; S102, using a filter to remove noise from the anxiety EEG dataset; S103, perform independent component analysis on the anxiety EEG dataset after noise removal; S104, based on the anxiety EEG data set after independent component analysis, arousal and valence scores were marked to obtain four categories: normal, mild anxiety, moderate anxiety, and severe anxiety; S105, segmenting the EEG signal according to the labeled anxiety EEG data set; S106, mapping the segmented anxiety EEG data set to [-1, 1] using maximum and minimum normalization to complete the preprocessing of the anxiety EEG data set; S107, dividing the preprocessed anxiety EEG data set into a training set and a test set.

3. The method for evaluating anxiety based on EEG signals driven by imbalanced data according to claim 1, characterized in that: The data enhancement model includes two adversarial learnings; the first adversarial learning includes a first encoder module and an encoding discriminator module; the second adversarial learning includes a second encoder module, a generator module and a generating discriminator module; The first encoder module is used to obtain training set samples and learn the distribution of the input training set samples in the latent space to obtain the latent vectors of the training samples in the latent space; The encoding discriminator module is used to obtain the latent vector z enc and a noise vector z randomly sampled from a standard normal distribution rand ; The generator module is used to generate a latent vector z according to the real sample enc , noise vector z rand And the corresponding category labels, output reconstructed samples and newly generated samples; The second encoder module is used to embed the label information into the latent vector z enc In a vector of the same size, combine the label information and the latent vector z enc Get a labeled latent vector and decode it back to its original dimension; The generated discriminator module is used to obtain the total validity score of true and false and class matching according to the sample pair, wherein the sample pair includes the real sample x real and the true sample label y real Sample pair, reconstructed sample x enc and reconstruct the sample label y enc Sample pairs and newly generated samples x rand And generate sample labels y rand Sample pairs.

4. The method for evaluating anxiety based on EEG signals driven by imbalanced data according to claim 3, characterized in that: The first encoder module and the second encoder module have the same structure, both comprising: three first temporal convolutional layer branches, a first channel attention, a first spatial convolutional layer connected to the first channel attention, a second channel attention connected to the first spatial convolutional layer, a first depthwise separable layer connected to the second channel attention, a first flattening layer connected to the first depthwise separable layer, and a first fully connected layer connected to the first flattening layer; The encoding discriminator module includes a plurality of second fully connected layers connected in sequence; The generator module includes a first label encoding layer, a third fully connected layer connected to the label encoding layer, a reshaping layer connected to the third fully connected layer, a fourth fully connected layer connected to the reshaping layer, a transposed convolution layer connected to the fourth fully connected layer, and a self-attention layer connected to the transposed convolution layer; The generative discriminator module includes three second temporal convolutional layers, a third channel attention, a second spatial convolutional layer connected to the third channel attention, a fourth channel attention connected to the second spatial convolutional layer, a second flattening layer connected to the fourth channel attention, and a fifth fully connected layer connected to the second flattening layer and the second label encoding layer.

5. The method for evaluating anxiety based on EEG signals driven by imbalanced data according to claim 4, characterized in that: The training process of the data enhancement model is as follows: A1. Obtain the imbalanced anxiety EEG training set; A2. In the first part of each training epoch, the training set samples are input into the first encoder module to learn the distribution of the input training set samples in the latent space and obtain the latent vector z of the training samples in the latent space. enc ; A3. The latent vector z enc and a noise vector z randomly sampled from a standard normal distribution rand Input to the encoding discriminator module to determine the authenticity of the input data; A4. According to the judgment result, train the first encoder module and the encoder discriminator module, and update the parameters of the first encoder module and the encoder discriminator module; A5. In the second part of each training epoch, the generator module receives the latent vector z of the real sample. enc , noise vector z rand And the corresponding category label, output the reconstructed sample and the newly generated sample; wherein the second encoder module embeds the label information into the latent vector z enc In a vector of the same size, combine the label information and the latent vector z enc Get a labeled latent vector and decode it back to its original dimension; A6. Input the sample pair into the generative discriminator module to obtain the total validity score of true and false and class matching. Among them, the sample pair includes the real sample x real and the true sample label y real Sample pair, reconstructed sample x enc and reconstruct the sample label y enc Sample pairs and newly generated samples x rand And generate sample labels y rand Sample pairs; A7. Use the total validity score of true and false and class matching to train the second encoder module, generator module and generated discriminator, and update the parameters of the second encoder module, generator module and generated discriminator to complete the training of the data enhancement model.

6. The method for evaluating anxiety based on EEG signals driven by imbalanced data according to claim 5, characterized in that: The expression of the loss function of the first adversarial learning is as follows: E(x real )=z enc Where L1 represents the loss function of the first adversarial learning, E(·) represents the operation of the first encoder module, and D E (·) represents the operation of the encoding discriminator module, Represents a given latent vector E(x real ) obeys the probability distribution P e Under the condition of real ), E(x real ) represents the potential vector of the real data output by the encoder, P e Represents the potential vector E(x) output by the encoder of the real data real ) obeys the probability distribution, Represents a given latent vector z rand Obey the probability distribution P t Under the condition of rand The conditional expectation of z rand represents a noise vector randomly sampled from a standard normal distribution, P t represents the noise vector z rand The probability distribution is obeyed, represents the weight of the gradient penalty, represents the gradient calculation, L GP represents the loss of the gradient penalty term, Indicates from z enc and z rand The data sampled from the mixed sample, represents the sampled latent vector, express The probability distribution that follows is E(x real ) and z enc Both represent latent vectors; The expression of the loss function of the second adversarial learning is as follows: L2=L real，enc +L real，rand G(z enc )=x enc Among them, L2 represents the loss function of the second adversarial learning, L real，enc Represents the real sample x real and reconstruct the sample x enc The distance metric between real，rand Represents the real sample x real and the newly generated sample x rand The distance measure between Represents the reconstructed sample G(z enc ) obeys the probability distribution P ze Under the condition of enc ), G(·) represents the operation of the generator, D G (·) represents the operation of generating the discriminator, G(z enc ) and x enc All represent reconstructed samples. Represents the real sample x real Obey the probability distribution P r Under the condition of real The conditional expectation of L GP-1 Represents the real sample x real and reconstruct the sample x enc The loss of the gradient penalty term during adversarial training corresponds to the loss of the gradient penalty term L GP , represents the reconstruction loss, λ1 represents the weight of the reconstruction loss, Represents the newly generated sample x rand Obey the probability distribution P zr Under the condition of rand The conditional expectation of G(z rand ) and x rand Both represent newly generated samples, L GP-2 Represents the real sample x real and the newly generated sample x rand Loss of the gradient penalty term during adversarial training.

7. The method for evaluating anxiety based on EEG signals driven by imbalanced data according to claim 1, characterized in that: The anxiety emotion assessment model in step S5 includes a spatiotemporal Transformer module, a dual-branch convolution module and a classification module connected in sequence; The spatiotemporal Transformer module includes a position encoding submodule, a time Transformer submodule and a space Transformer submodule both connected to the position encoding submodule, and a feature fusion submodule both connected to the time Transformer submodule and the space Transformer submodule; The dual-branch convolution module includes a temporal convolution submodule, a fifth channel attention, a deep convolution submodule connected to the fifth channel attention, and a sixth channel attention connected to the deep convolution submodule; wherein the temporal convolution submodule includes three third temporal convolution layers with different kernel sizes, and the deep convolution submodules each include two branches, each branch including a spatial convolution layer for obtaining multi-scale spatial features from a feature map and a spatiotemporal convolution layer for learning to fuse multi-scale spatiotemporal features; The classification module includes a third flattening layer, a sixth fully connected layer and a softmax layer which are connected in sequence.

8. The method for evaluating anxiety based on EEG signals driven by imbalanced data according to claim 7, characterized in that: The position encoding submodule is used to add position information to the EEG signal; The temporal Transformer submodule is used to extract the correlation between different time points of the multi-channel EEG signals according to the EEG signals with added position information to obtain the temporal features; The spatial Transformer submodule is used to extract the correlation between different channels of the multi-channel EEG signal according to the EEG signal with added position information to obtain the spatial feature; The feature fusion submodule is used to fuse the temporal features and the spatial features to obtain a feature vector F; The time convolution submodule is used to obtain the multi-scale time features of the multi-channel EEG signal according to the feature vector F; The deep convolution submodule is used to extract the spatiotemporal features of the EEG signal according to the multi-scale time features, and fuse the multi-scale time features with the spatiotemporal features by using a summation operation to obtain the deep spatiotemporal features of the EEG signal; The third flattening layer is used to convert the multi-dimensional deep spatiotemporal features output by the dual-branch convolution module into one-dimensional features; The sixth fully connected layer is used to perform linear transformation on the deep spatiotemporal features of the EEG signal to obtain the prediction probability; The softmax layer is used to output a predicted classification result according to the predicted probability, wherein the classification results include normal, mild anxiety, moderate anxiety and severe anxiety.

9. The method for evaluating anxiety based on EEG signals driven by imbalanced data according to claim 8, characterized in that: The training process of the anxiety emotion assessment model is as follows: B1. Obtain the balance anxiety EEG training set; B2. Input the balance anxiety EEG training set into the spatiotemporal Transformer module and fuse it to obtain the feature vector F, in which the EEG signals are arranged in both space and time. B3, using the fused feature vector F, using a double-branch convolution module to obtain the deep spatiotemporal features of the EEG signal; B4. According to the deep spatiotemporal characteristics of the EEG signal, the classification module is used to obtain the prediction probability, and the prediction classification result is obtained according to the prediction probability; B5. Use the predicted classification results and the true classification labels to calculate the value of the loss function of the anxiety emotion assessment model, and back-propagate the gradient according to the value of the loss function to update the parameters of the anxiety emotion assessment model until convergence, thus completing the training of the anxiety emotion assessment model.

10. The method for assessing anxiety based on EEG signals driven by imbalanced data according to claim 9, characterized in that: The loss function of the anxiety assessment model is expressed as follows: Among them, L(x,y) represents the loss function of the anxiety assessment model, N represents the batch size, i represents the i-th sample, and l i represents the loss of the i-th sample, y represents the true category label, j represents the j-th category, C represents the number of categories, and x i,j It represents the probability that the i-th sample belongs to the j-th category.

Images