A method and device for multi-listener electroencephalogram signal generation and cross-listener emotion recognition

Abstract

The present application relates to a kind of multi-listener electroencephalogram generation and cross-listener emotion recognition method, comprising the following steps: collecting listener electroencephalogram under different emotional speech induction;Different speech signals are preprocessed and data enhanced;Establish the generation model of electroencephalogram, and the training of generation model is executed;Generation model is executed, generates multiple listener electroencephalogram, extracts the ERP waveform in the generated multiple listener electroencephalogram;Classification model is constructed, and the ERP waveform in the generated multiple listener electroencephalogram is classified emotionally.The present application solves the problem of cross-listener emotion recognition by using ERP waveform for cross-listener recognition, extracts the energy information and timing information of superimposed ERP waveform, realizes the accurate classification of speech-induced electroencephalogram, and cross-subject emotion recognition reaches better recognition effect.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of electroencephalogram signal processing, and particularly relates to a method for generating electroencephalogram signals of multiple listeners and cross-listener emotion recognition. BACKGROUND

[0002] With the significant progress of AI and brain-computer interface (BCI) technology, the encoding mechanism of the brain to external stimuli is also being explored. Electroencephalogram (EEG) signals are a general reflection of the electrical physiological activity of nerve cells in the brain and can reflect the thinking activity of the human brain. It is not affected by subjective factors and can effectively shield task-irrelevant induced interference, and the recognition result has strong stability and anti-interference. Electroencephalogram signals contain all the information of human brain cognition, and the recognition of emotions by electroencephalogram signals is a direct result of human brain cognition. Therefore, the result of emotion recognition of speech by electroencephalogram signals is the true emotional state of human brain intelligence.

[0003] In actual human-computer interaction, the emotional speech of the speaker can be easily captured, but the collection of electroencephalogram signals is relatively cumbersome, and it is difficult to collect the electroencephalogram signals of the listener for accurate emotion prediction in actual situations. SUMMARY

[0004] The purpose of the application is to provide a method and device for generating electroencephalogram signals of multiple listeners and cross-listener emotion recognition, which can solve the problem of difficulty in collecting electroencephalogram signals of the listener for accurate prediction.

[0005] To achieve the above purpose, the technical solution adopted by the application is:

[0006] A method for generating electroencephalogram signals of multiple listeners and cross-listener emotion recognition, comprising the following steps:

[0007] Collecting electroencephalogram signals of the listener under the induction of different emotional speech;

[0008] Pretreating and data enhancing different speech signals;

[0009] Establishing a generation model of electroencephalogram signals and performing training of the generation model;

[0010] Performing the generation model to generate multiple electroencephalogram signals of the listener and extracting ERP waveforms in the generated multiple electroencephalogram signals of the listener;

[0011] Constructing a classification model to perform emotion classification on the generated electroencephalogram signals of the listener.

[0012] Preferably, the collecting the brain electrical signals of the listener further comprises the following steps: the listener respectively performs emotional judgment and semantic repetition on the same emotional speech, and simultaneously collects the brain electrical signals of the listener, wherein the emotional judgment refers to the identification and judgment of the emotional category of the speech signal, and the semantic repetition refers to the repetition of the speech content of the speech signal.

[0013] Preferably, the training of the generation model comprises the following steps: adopting a conditional generative adversarial network as the generation model, preprocessing the collected brain electrical signals of the listener; inputting the data-enhanced speech signal into the generator of the conditional generative adversarial network; the generator generates the brain electrical signals of the listener; the discriminator of the conditional generative adversarial network discriminates the preprocessed brain electrical signals of the listener and the generated brain electrical signals of the listener, and performs training iteration.

[0014] The execution of the generation model comprises the following steps: data-enhancing the other speech signals into a plurality of speech observation signals, and inputting the enhanced plurality of speech observation signals into the generation model to generate a plurality of brain electrical signals of the listener.

[0015] Preferably, the speech signal preprocessing is a double-threshold endpoint detection method of short-time energy and zero-crossing rate on the speech signal.

[0016] The speech signal data enhancement comprises the following steps: adopting a random observation matrix to observe the speech signal multiple times to obtain a plurality of observation values of the speech signal; and the dimension of the observation value is less than the dimension of the speech signal.

[0017] Preferably, the method further comprises the following steps:

[0018] The generated plurality of brain electrical signals of the listener are superimposed to extract an ERP waveform; the ERP waveform is divided into a plurality of same signal segments, and the number of sliding windows is calculated as follows:

[0019]

[0020] Wherein, S is the length of the generated brain electrical signals of the listener, m is the sliding window step length, and l is the sliding window length.

[0021] The segmented ERP waveform is decomposed into five frequency bands of theta rhythm (4-8Hz), alpha rhythm (8-12Hz), low beta rhythm (12-16Hz), high beta rhythm (16-25Hz) and gamma rhythm (25-45Hz).

[0022] Preferably, the classification model is composed of a convolutional neural network (CNN) and a long short-term memory network (LSTM); the convolutional neural network is used to extract the frequency band energy features of the ERP waveform; and the long short-term memory network comprises a forgetting gate, an input gate, a cell update unit and an output gate.

[0023] The forget gate determines which information from the previous cell is discarded or retained, and is formulated as:

[0024] f t = σ(W f q t + U f h t-1 + b f )

[0025] The input gate determines which information is used for updating the cell state:

[0026] i t = σ(W i q t + U i h t-1 + b i )

[0027]

[0028] The cell update unit point-wise multiplies the cell state of the previous layer with the forget vector to update the cell state:

[0029]

[0030] The output gate is used to determine the value of the next hidden state, which contains the information of the previous input:

[0031] o t = σ(W o q t + U o h t-1 + b o )

[0032] h t = o t ⊙ σ(c t )

[0033] where σ is the sigmoid logistic regression function, q t is the feature vector extracted by the CNN layer, h t is the feature vector extracted by the LSTM layer, f t is the forget gate, i t is the input gate, o t is the output gate, W f , W i , W c , W o , U f , U i , U c , U o are transformation matrices, and b f , bi , b c , b o is a bias term.

[0034] Preferably, the method further comprises the following steps:

[0035] The feature vector of the classification model is input into a full connection layer with a softmax as an activation function, so as to obtain a probability that the generated listener electroencephalogram segment is predicted as the i-th emotion, wherein the probability is:

[0036]

[0037] wherein W is a transformation matrix, b is a bias term, K is the number of emotion categories,

[0038] The classification model is trained through cross-entropy loss, and the loss function is defined as:

[0039]

[0040] wherein y is a unique vector of the real label of the electroencephalogram segment.

[0041] According to another aspect of the present application, a computer readable storage medium is also provided, which stores a computer program, and the computer program is executed by a processor to perform the steps of the above-mentioned method for generating multi-listener electroencephalogram signals and cross-listener emotion recognition.

[0042] According to another aspect of the present application, a multi-listener electroencephalogram signal generation and cross-listener emotion recognition device is also provided, comprising:

[0043] a memory for storing a software application,

[0044] a processor for executing the software application, and each program of the software application corresponds to the steps of the above-mentioned method for generating multi-listener electroencephalogram signals and cross-listener emotion recognition.

[0045] The present application has the following advantages:

[0046] 1. The method for collecting listener electroencephalogram signals realizes the collection of electroencephalogram signals in terms of emotion and semantics, and is used for multi-modal different data analysis of electroencephalogram and voice.

[0047] 2、The application adopts a random observation matrix to realize enhancement of voice data, uses a random Gaussian matrix to observe the voice of a speaker to obtain multiple enhanced voice observation signals, increases the randomness of input data, obtains multiple listener electroencephalogram signals, solves the problem of model training using small sample data, and also realizes dimension reduction processing of the signals, so that the running cost of the algorithm is saved, and effective features in the original signals are not lost.

[0048] 3、The application models the mapping relationship between the voice and the electroencephalogram signal by using a conditional generative adversarial network, realizes generation of the electroencephalogram signal from the voice signal, and is beneficial to the target of predicting the emotion of the voice signal by the electroencephalogram signal.

[0049] 4、ERP waveform is used for cross-listener recognition to solve the problem of cross-listener emotion recognition. The ERP waveform is superimposed on the electroencephalogram signal generated by the enhancement strategy to obtain more obvious ERP waveform. The energy information and timing information of the superimposed ERP waveform are extracted to realize accurate emotion classification of the voice-induced electroencephalogram signal, and good recognition effect is achieved in cross-subject emotion recognition. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 It is a process flowchart of the application;

[0051] Figure 2 (A) is a data enhancement flowchart based on a random observation matrix, (B) is a training flowchart of a conditional generative adversarial network, and (C) is an electroencephalogram data generation flowchart based on the conditional generative adversarial network;

[0052] Figure 3 (A) is an electroencephalogram classification model structure diagram, and (B) is a long short-term memory network structure diagram;

[0053] Figure 4 It is a cross-listener emotion recognition average accuracy diagram of real EEG / ERP and generated EEG / ERP. DETAILED DESCRIPTION

[0054] The application will be further described below in combination with the drawings and specific embodiments.

[0055] The terms involved in the application are defined as follows:

[0056] "Multiple listeners" refer to multiple listeners.

[0057] "Cross-listener" refers to cross-subject recognition, that is, the data for training and the data to be recognized are not the same data.

[0058] As shown in Figure 1 The application is a multiple-listener electroencephalogram signal generation and cross-listener emotion recognition method, which comprises the following steps:

[0059] S1, brain electrical signal acquisition: collect different emotional speech-induced listener brain electrical signals;

[0060] S2, speech data enhancement: pre-processing and data enhancement of different speech signals;

[0061] Specifically, in step S1, the collection of listener brain electrical signals further comprises the following steps:

[0062] S11, the listener performs speech emotion judgment and semantic repetition on the same emotional speech respectively, and simultaneously collects the listener brain electrical signals, wherein the emotion judgment refers to the listener's identification and judgment of the emotional category of the speech signal, and the semantic repetition refers to the listener's repetition of the speech content of the speech signal;

[0063] Specifically, in step S2, the speech signal pre-processing is a double-threshold endpoint detection method using short-time energy and zero-crossing rate;

[0064] In some embodiments, the emotional speech is selected from the CASIA emotional speech database, including angry, happy, sad and surprised speech signals, 30 sentences for each emotion, and the semantics of each emotional speech is the same, denoted as real speech signal;

[0065] Meanwhile, the double-threshold endpoint detection method using short-time energy and zero-crossing rate is used for the selected real speech signal, which detects the starting endpoint of the speech signal, removes the silent part in front of the speech signal, and retains the complete speech signal;

[0066] During the double-threshold endpoint detection of the real speech signal, the speech with different durations is set as a fixed duration of 3000ms, a notebook computer is used as a stimulus presentation device, an external microphone is used as a speech recording device, and the sampling rate is 16000Hz;

[0067] The listener listens to each speech twice while their EEG signals are collected simultaneously. Each speech signal employs a dual-task mode. In the first listening, the listener must judge the emotion of the speech. Each emotion corresponds to a different number key. By selecting the corresponding emotion key, the listener can judge the emotion of the speech without needing to understand the content; they only need to discern the rhythm and emotion. In the second listening, the listener carefully listens to the content of the speech and then completely retells and records the speech. They do not need to retell the emotion; instead, they need to accurately reproduce the content of the speech signal word by word. During the semantic retelling process, the listener's retelling is recorded. This process continues until the collection is complete. The collection process uses a pseudo-random arrangement, meaning that speech signals with different emotions do not appear consecutively. Through the above collection method, EEG signals are collected from both emotional and semantic aspects for multimodal data analysis of EEG and speech.

[0068] This invention selects 16 channels as regions of interest (ROIs) in six brain regions—prefrontal (left / right), central (left / left), and parietal (left / right)—namely Fz, FC3, FC1, FCz, FC4, FC2, C3, C1, CZ, C4, C2, CP3, CP1, CPZ, CP2, and CP4.

[0069] Specifically, in step S2, the voice signal data enhancement includes the following steps:

[0070] S21. The speech signal is observed multiple times using a random observation matrix to obtain multiple observation values ​​of the speech signal; the dimension of the observation values ​​is smaller than the dimension of the speech signal.

[0071] More specifically, a random Gaussian matrix Φ is used to perform perceptual observation on the preprocessed N-dimensional real speech signal X1, such as... Figure 2 As shown in (A), an M×N dimensional Gaussian matrix is ​​used to perform perceptual observations on the real speech signal, where M << N, resulting in multiple observation values ​​Y1 = Φ with a compression ratio of M / N that are much smaller than the original signal dimension. j X1; The enhanced speech signal is obtained by setting multiple different Gaussian matrices, that is, for b different speech signals, a (a≥2) different Gaussian matrices Φ are set. j b×a different enhanced speech signals corresponding to the speech signal are obtained;

[0072] In some embodiments, the number of observations is the same as the number of listeners, thereby expanding the amount of speech signal.

[0073] S3. EEG signal generation: Establish an EEG signal generation model and train the generation model.

[0074] Specifically, in step S3, the training of the generation model comprises the following steps:

[0075] S31, adopt a conditional generative adversarial network as a generation model; pre-process the speech-induced listener electroencephalogram; input the data-enhanced speech signal into the generator of the conditional generative adversarial network; the generator generates the listener electroencephalogram; the discriminator of the conditional generative adversarial network discriminates the pre-processed listener electroencephalogram and the generated listener electroencephalogram, and performs training iteration;

[0076] Specifically, the listener electroencephalogram pre-processing is to select the whole brain average as a reference, and to perform 0.5-30Hz filtering processing on the generated listener electroencephalogram by Fourier transform, and then to perform artifact removal processing;

[0077] Wherein, the artifact removal is a method of removing motion and electrooculogram artifact average values exceeding ±100μV;

[0078] Specifically, the conditional generative adversarial network is a generation model trying to find a latent data distribution, which is composed of a generator G and a discriminator D, and the enhanced speech signal and the original speech signal obtained in step S21 are input into the generator G, the generator encodes the input speech signal, and tries to generate samples consistent with the speech-induced electroencephalogram to deceive the discriminator D, and the discriminator D tries to distinguish the electroencephalogram generated by the generator G from the speech-induced electroencephalogram, and the training process is to fix one of the networks, update the parameters of the other network, and alternately iterate, so that the error of the other party is maximized, and finally the generator G estimates the distribution of sample data, and the process is as shown in Figure 2 (B), in the training process, the loss function needs to be optimized and adjusted to balance the generator and the discriminator;

[0079] The loss function is as follows:

[0080]

[0081] Wherein, a=c=1, b=0, is an expectation, and its essence is the maximum likelihood estimation of the generation model under the known observation data;

[0082] The training process of the conditional generative adversarial network is the mutual confrontation of the generator and the discriminator, and the loss is maximized from the perspective of the discriminator, and the loss is minimized from the perspective of the generator;

[0083] In the training process, a similarity threshold is defined, and the similarity threshold is 0.97. When the similarity between the sample generated by the generator and the real data reaches the threshold, that is, when the generator can generate synthesized data similar to the brain electrical signals induced by the voice, the discriminator network cannot distinguish which data is real data and which data is generated by the generator, and it is considered that the generation model has been trained.

[0084] After the conditional generative adversarial network is trained, the voice signal is input, and the listener's brain electrical signal is generated by using the mapping relationship between the speaker's voice signal and the brain electrical signal induced by the voice. The process is as shown in Figure 2 (C).

[0085] S4, brain electrical feature extraction: executing the generation model to generate a plurality of listener brain electrical signals, and extracting the ERP waveform in the generated plurality of listener brain electrical signals;

[0086] Specifically, the voice signals of other speakers are subjected to one-to-many data enhancement by the data enhancement method of step S21, and are enhanced into a plurality of voice observation signals. The enhanced plurality of voice observation signals are input into the trained generation model to generate a plurality of listener brain electrical signals, which enhances the prediction ability of the generated brain electrical signals on the listener's emotion, and makes the listener brain electrical signals have the same emotion.

[0087] Among them, the voice signal of other speakers is a voice signal different from the voice signal selected in step S1,

[0088] Specifically, the following steps are further included:

[0089] S41, superimposing the generated plurality of listener brain electrical signals to obtain the ERP waveform; dividing the ERP waveform into a plurality of same signal segments, and the number of sliding windows is calculated as follows:

[0090]

[0091] Among them, S is the length of the generated listener brain electrical signal, m is the sliding window step length, and l is the sliding window length. S takes 3 seconds (3000 data), m takes 16 data segments, and l takes 2 seconds.

[0092] The segmented ERP waveform is decomposed into five frequency bands of theta rhythm (4-8 Hz), alpha rhythm (8-12 Hz), low beta rhythm (12-16 Hz), high beta rhythm (16-25 Hz) and gamma rhythm (25-45 Hz), and the frequency band energy of each ERP waveform segment is calculated and flattened as the input of the classification model.

[0093] More specifically, the generated multiple brain electrical signals are superimposed to extract the ERP waveform corresponding to the speech signal of other speakers, the ERP waveform is locked from the stimulus start time, and the waveform between 0ms at the stimulus start and 3000ms after the stimulus start is averaged.

[0094] S5, cross-listener brain electrical signal emotion recognition: build a classification model to classify the generated listener brain electrical signals.

[0095] Specifically, as shown in Figure 3 (A), the classification model is composed of a convolutional neural network CNN and a long short-term memory network LSTM; the convolutional neural network is composed of a one-dimensional convolutional neural network, which extracts deeper feature information in the frequency band energy feature information of the ERP waveform in step S41;

[0096] The long short-term memory network includes a forgetting gate, an input gate, a cell update unit and an output gate, which extracts the time sequence information from the frequency band energy features in the convolutional neural network,

[0097] As shown in Figure 3 (B), the forgetting gate determines which information from the previous cell element to discard or retain, and its formula is:

[0098] f t =σ(W f q t +U f h t-1 +b f )

[0099] The input gate determines which is used for cell state update:

[0100] i t =σ(W i q t +U i h t-1 +b i )

[0101]

[0102] The cell update unit point-wise multiplies the cell state of the previous layer to update the cell state:

[0103]

[0104] The output gate is used to determine the value of the next hidden state, which contains the information of the previous input:

[0105] o t =σ(W o q t +U o ht-1 +b o )

[0106] h t =o t σ(c t )

[0107] wherein, σ is a sigmoid logistic regression function, q t is a feature vector extracted after a CNN layer, h t is a feature vector extracted after an LSTM layer, f t is a forgetting gate, i t is an input gate, o t is an output gate, W f , W i , W c , W o , U f , U i , U c , U o is a transformation matrix, b f , b i , b c , b o is a bias term;

[0108] wherein, in the classification model, three convolutional layers, one batch normalization layer, two LSTM layers and one fully connected layer are set. The convolutional layers are one-dimensional convolutional layers used to obtain time sequence information, and each convolutional kernel has 64. The lengths of the three convolutional kernels are 15, 3 and 3, and the step length is 2 in the first and third convolutional layers. A relu activation function is set after the first convolutional layer, and a batch normalization layer is set after the convolutional operation is completed. The output dimension of the LSTM network is set to 512 and 128, and half of the hidden neurons are discarded in the first LSTM network;

[0109] wherein, after the superimposed EPR waveform is input into the classification model, the convolutional neural network extracts the frequency band energy features, and the long short-term memory network extracts the time sequence information from the convolutional neural network to perform emotion classification on the generated listener electroencephalogram signal, predict the emotion category, and output the classification result. More specifically, different ERP waveforms correspond to different emotions, and according to different ERP waveforms, emotions are classified and divided into different categories;

[0110] Specifically, the method further comprises the following steps:

[0111] S51, the feature vector of the classification model is input into a fully connected layer with a softmax as an activation function, and the probability that the generated listener electroencephalogram signal segment is predicted as the i-th emotion is obtained as follows:

[0112]

[0113] out = Wh n +b

[0114] wherein W is a transformation matrix, b is a bias term, K is the number of emotion categories,

[0115] The classification model is trained by cross-entropy loss, and the loss function is defined as:

[0116]

[0117] wherein y is a unique vector of the real label of the brain electrical segment.

[0118] The prediction result obtained by inputting the ERP waveform of the speech-induced listener brain electrical signal and the generated listener brain electrical signal into the classification model, wherein the compression ratio of data enhancement = 0.3, as shown in the following table: Figure 4 The speech-induced listener brain electrical signal is the real EEG, the generated listener brain electrical signal is the generated EEG, the ERP waveform of the speech-induced listener brain electrical signal is the real ERP, and the ERP waveform of the generated listener brain electrical signal is the generated ERP,

[0119] In cross-listener emotion recognition, the recognition rate of the generated ERP is increased by nearly 30% compared with the real ERP, and the recognition rate of the generated listener brain electrical signal is much higher than that of the speech-induced brain electrical signal in the recognition rate of anger, sadness, happiness and surprise, which further shows that the ERP waveform feature is more excellent than the speech-induced brain electrical signal in cross-listener emotion recognition, and confirms that the noise of the brain electrical signal generated by the generation model is smaller, and the ERP waveform feature is more obvious.

[0120] According to another aspect of the present application, a computer readable storage medium having a computer program stored thereon is also provided, the computer program being executed by a processor to perform the steps of the above-mentioned method for generating multi-listener brain electrical signals and cross-listener emotion recognition.

[0121] According to another aspect of the present application, a multi-listener brain electrical signal generation and cross-listener emotion recognition device is also provided, comprising:

[0122] a memory for storing a software application,

[0123] a processor for executing the software application, each program of the software application corresponding to the steps of the above-mentioned method for generating multi-listener brain electrical signals and cross-listener emotion recognition.

Claims

1. A method for generating EEG signals from multiple listeners and recognizing emotions across listeners, characterized in that, Includes the following steps: Collecting EEG signals from listeners induced by different emotional speech patterns; Preprocessing and data enhancement of different speech signals; Establish a model for generating EEG signals and train the model. The generative model is executed to generate multiple listener EEG signals, and the ERP waveforms are extracted from the generated multiple listener EEG signals. A classification model was constructed to classify the emotions of ERP waveforms in multiple generated EEG signals from listeners. The training of the generative model includes the following steps: using a conditional generative adversarial network (GAN) as the generative model, preprocessing the collected listener's EEG signals; inputting the data-enhanced speech signal into the generator of the GAN; generating the listener's EEG signals; and the discriminator of the GAN distinguishing between the preprocessed listener's EEG signals and the generated listener's EEG signals, and performing training iterations. The execution of the generative model includes the following steps: data augmentation of other speech signals to generate multiple speech observation signals, and inputting the augmented multiple speech observation signals into the generative model to generate multiple listener EEG signals.

2. The method for generating multi-auditor EEG signals and recognizing cross-auditor emotions according to claim 1, characterized in that, The process of collecting listener EEG signals also includes the following steps: the listener performs voice emotion judgment and semantic paraphrasing on the same emotional speech, and the listener's EEG signals are collected simultaneously. The emotion judgment refers to the listener identifying and judging the emotion category of the speech signal, and the semantic paraphrasing refers to the listener paraphrasing the speech content of the speech signal.

3. The method for generating EEG signals from multiple listeners and recognizing emotions across listeners according to claim 1, characterized in that, The speech signal preprocessing method employs a dual-threshold endpoint detection method based on short-time energy and zero-crossing rate for the speech signal. The speech signal data enhancement includes the following steps: using a random observation matrix to observe the speech signal multiple times to obtain multiple observation values ​​of the speech signal; the dimension of the observation values ​​is smaller than the dimension of the speech signal.

4. The method for generating multi-auditor EEG signals and recognizing cross-auditor emotions according to claim 1, characterized in that, It also includes the following steps: Multiple listener EEG signals were superimposed to extract the ERP waveform. The ERP waveform was then divided into multiple identical signal segments, and the number of sliding windows was calculated as follows: ， Where S is the length of the generated listener's EEG signal, m is the sliding window step size, and l is the sliding window length; The segmented ERP waveform is decomposed into five frequency bands: theta rhythm (4-8Hz), alpha rhythm (8-12Hz), low beta rhythm (12-16Hz), high beta rhythm (16-25Hz), and gamma rhythm (25-45Hz).

5. The method for generating EEG signals from multiple listeners and recognizing emotions across listeners according to claim 1, characterized in that, The classification model consists of a convolutional neural network (CNN) and a long short-term memory network (LSTM); the CNN is used to extract the frequency band energy features of the ERP waveform; the LSTM network includes a forget gate, an input gate, a cell renewal unit, and an output gate. The forget gate determines which information from the previous cell is discarded or retained, and its formula is as follows: ， The input gate determines which updates are used for the cell state: ， ， The cell update unit updates the cell state by multiplying the cell state of the previous layer point by point by the forget vector: ， The output gate is used to determine the value of the next hidden state, which contains the information previously input: ， ， in, It is the sigmoid logistic regression function. It is the feature vector extracted after passing through the CNN layer. It is the feature vector extracted after passing through the LSTM layer. It is the Gate of Oblivion. It's an input gate. It's an output gate. , , , , , , , It is a transformation matrix. , , , It is a bias term.

6. The method for generating EEG signals from multiple listeners and recognizing emotions across listeners according to claim 5, characterized in that, It also includes the following steps: The feature vector of the classification model is input into a fully connected layer with softmax as the activation function, and the resulting ERP waveform segments from multiple listener EEG signals are predicted as the first... The probability of a sentiment is: ， ， in, The generated EEG signals of the listener belong to The probability of a class is given by W, where W is the transformation matrix, b is the bias term, and K is the number of sentiment classes. The classification model is trained using cross-entropy loss, and the loss function is defined as: ， in, A unique vector representing the true label of an EEG segment.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it performs the steps of the method for generating EEG signals from multiple listeners and recognizing emotions across listeners as described in any one of claims 1 to 6.

8. A device for generating EEG signals from multiple listeners and recognizing emotions across listeners, characterized in that, include: Memory, used to store software applications. A processor for executing the software application, wherein each program of the software application correspondingly performs the steps of the method for generating EEG signals from multiple listeners and recognizing emotions across listeners as described in any one of claims 1 to 6.

Images