Emotion mode high-resolution extraction method and device based on electroencephalogram signals

Abstract

The invention discloses an emotion mode high-resolution extraction method and device based on electroencephalogram signals. The method comprises the steps that an electroencephalogram signal original data set is obtained; the electroencephalogram signal original data set comprises an electroencephalogram signal sequence and corresponding emotion label information; preprocessing the electroencephalogram signal original data set to obtain an electroencephalogram signal training data set; training an emotion mode extraction model by using the electroencephalogram signal training data set to obtain a trained emotion mode extraction model; and processing the acquired electroencephalogram signals by using the trained emotion mode extraction model to obtain an emotion mode extraction result. According to the method, the efficiency and accuracy of emotion recognition are improved by utilizing the electroencephalogram signals under the fine-grained frequency domain resolution.

Description

Technical Field

[0001] The present invention relates to the fields of electroencephalogram (EEG) signal processing and artificial intelligence, and in particular to a method and device for extracting emotional patterns with high resolution based on EEG signals. Background Art

[0002] EEG signals are multi-channel time series that contain a large amount of central nervous system information, which is directly related to high-level cognitive processes, including emotions. The frequency domain is one of the key feature dimensions of EEG signals. In the traditional EEG signal analysis framework, EEG signals are often divided into 4 to 5 classic frequency bands with extremely low frequency domain resolution. Following the classical EEG analysis framework, many outstanding research works have made many advances. However, people's understanding of how emotions exist in electroencephalogram signals (EEG) at fine-grained frequency domain resolution is still shallow, and the limited resolution in the spectral domain may hinder our model from achieving higher performance in recognizing emotions.

[0003] Existing emotion recognition methods do not consider the impact of EEG signals at fine-grained frequency domain resolution on emotion recognition tasks. They only analyze EEG signals in 4 to 5 classic frequency bands, ignoring the problem that the limited resolution in the spectral domain may hinder our model from achieving higher performance in recognizing emotions, resulting in reduced emotion recognition ability and low accuracy of emotion recognition. Summary of the invention

[0004] The present invention mainly solves the problem that the existing emotion recognition methods do not consider the influence of EEG signals at fine-grained frequency domain resolution on emotion recognition tasks, resulting in reduced emotion recognition ability and low accuracy of emotion recognition. The present invention discloses a method and device for high-resolution extraction of emotion patterns based on EEG signals.

[0005] In a first aspect of the embodiments of the present application, a method for extracting high-resolution emotion patterns based on EEG signals is disclosed, comprising:

[0006] S1, obtaining an original EEG signal data set; the original EEG signal data set includes an EEG signal sequence and corresponding emotion label information;

[0007] S2, preprocessing the original EEG signal data set to obtain an EEG signal training data set;

[0008] S3, using the EEG signal training data set to train the emotion pattern extraction model to obtain a trained emotion pattern extraction model;

[0009] S4, using the trained emotion pattern extraction model to process the collected EEG signal to obtain an emotion pattern extraction result.

[0010] The emotion pattern extraction model includes: a preprocessing module, a frequency domain encoding module, a space encoding module and a classifier module;

[0011] The preprocessing module is used to preprocess the received EEG signal sequence to obtain a spatial signal set and a frequency domain signal set;

[0012] The frequency domain coding module is connected to the preprocessing module and the feature fusion module respectively, and is used to perform frequency domain coding processing on the received frequency domain signal set to obtain a frequency domain coded signal;

[0013] The spatial coding module is connected to the preprocessing module and the feature fusion module respectively, and is used to perform spatial coding processing on the received spatial signal set to obtain a spatially coded signal;

[0014] The classifier module is connected to the space coding module and the frequency domain coding module, and is used to classify the received frequency domain coding signal and space coding signal to obtain the emotion pattern recognition result.

[0015] The preprocessing of the original EEG signal data set to obtain an EEG signal training data set includes:

[0016] S21, performing data cleaning processing on the original EEG signal data set to obtain a cleaned data set;

[0017] S22, performing pattern detection processing on the cleaned data set to obtain an EEG signal training data set.

[0018] The performing pattern detection processing on the cleaned data set to obtain an EEG signal training data set includes:

[0019] S221, for each type of data attribute of the cleaned data set, using the data collection information of the data as an independent variable and the data value of the data as a dependent variable, perform autoregression-sliding average modeling to obtain regression models of the data attributes of each type;

[0020] S222, using the regression model, calculating and processing the independent variable to obtain a regression data value; determining whether the absolute value of the difference between the regression data value and the corresponding dependent variable value is greater than a set first regression discrimination threshold; if it is greater than the first regression discrimination threshold, deleting the data from the cleaned data set; if it is less than or equal to the first regression discrimination threshold, not processing the data;

[0021] S223, performing fusion processing on all data of the cleaned data set after executing S221 to S222 to obtain an EEG signal training data set.

[0022] The preprocessing module preprocesses the received EEG signal sequence to obtain a spatial signal set and a frequency domain signal set, including:

[0023] Performing spectrum conversion processing on the received EEG signal sequence of each electrode to obtain the EEG spectrum signal of each electrode;

[0024] The EEG spectrum signals of all electrodes are spliced ​​and processed to obtain EEG space-spectrum image information;

[0025] The EEG space-spectrum image information is processed in the frequency domain to obtain a frequency domain signal set; the frequency domain signal set X t1 The expression is

[0026] X t1 =[a1, a2, …, a F ],

[0027] Among them, a i is the frequency domain signal of the i-th frequency band, and F is the total number of frequency bands;

[0028] The EEG space-spectrum image information is spatially represented to obtain a spatial signal set; the spatial signal set X t2 The expression is

[0029]

[0030] Among them, b i is the frequency domain signal of the signal collected by the i-th electrode, T represents the transposition operation, and C is the total number of electrodes.

[0031] The frequency domain coding module includes an input module, a mask processing module, a position coding module, a frequency domain encoder, and a frequency domain decoder;

[0032] The mask processing module is connected to the input module and the position encoding module, and is used to perform mask processing on the signal sequence received by the input module according to a preset ratio value to obtain a masked sequence; the masked sequence has a corresponding signal sequence;

[0033] The position encoding module is connected to the frequency domain encoder and is used to perform position encoding processing on the masked sequence to obtain a first feature sequence;

[0034] The frequency domain encoder is connected to the frequency domain decoder and is used to encode the first feature sequence to obtain a frequency domain coded signal;

[0035] The frequency domain decoder is used to decode the frequency domain coded signal to obtain a frequency domain decoded signal;

[0036] The spatial encoding module includes an input module, a mask processing module, a position encoding module, a spatial encoder, and a spatial decoder;

[0037] The mask processing module is connected to the input module and the position encoding module, and is used to perform mask processing on the signal sequence received by the input module according to a preset ratio value to obtain a masked sequence;

[0038] The position encoding module is connected to the spatial encoder and is used to perform position encoding processing on the masked sequence to obtain a second feature sequence;

[0039] The spatial encoder is connected to the spatial decoder and is used to encode the second feature sequence to obtain a spatially coded signal;

[0040] The spatial decoder is used to decode the spatially encoded signal to obtain a spatially decoded signal.

[0041] The training process of the emotion pattern extraction model includes:

[0042] Pre-training the frequency domain coding module and the space coding module to obtain a pre-trained emotion pattern extraction model;

[0043] Initialize the number of training iterations;

[0044] Using the EEG signal sequence in the EEG signal training data set as input data, inputting it into the pre-trained emotion pattern extraction model to obtain an emotion prediction value;

[0045] Performing difference calculation processing on the emotion label information and the emotion prediction value of the input data to obtain a difference value;

[0046] Determine whether the difference value satisfies a convergence condition, and obtain a first determination result;

[0047] When the first judgment result is no, judging whether the training iteration number value is equal to the training number threshold, and obtaining a second judgment result;

[0048] When the second judgment result is no, determining that the model training state does not meet the training termination condition;

[0049] When the second judgment result is yes, determining that the model training state satisfies the training termination condition;

[0050] When the first judgment result is yes, determining that the model training state satisfies a training termination condition;

[0051] When the model training state does not meet the training termination condition, the parameter update model is used to update the parameters of the classifier module, the training iteration number value is increased by 1, and the execution is triggered to use the EEG signal sequence in the EEG signal training data set as input data;

[0052] When the model training state satisfies the training termination condition, the training process of the pre-trained emotion pattern extraction model is completed to obtain a trained emotion pattern extraction model.

[0053] In a second aspect of the embodiment of the present application, a device for extracting high-resolution emotion patterns based on EEG signals is disclosed, the device comprising:

[0054] A memory storing executable program code;

[0055] a processor coupled to the memory;

[0056] The processor calls the executable program code stored in the memory to execute the method for high-resolution extraction of emotion patterns based on EEG signals.

[0057] In a third aspect of an embodiment of the present application, a computer-storable medium is disclosed, wherein the computer-storable medium stores computer instructions, and when the computer instructions are called by a computer, they are used to execute the method for high-resolution extraction of emotion patterns based on EEG signals.

[0058] In a fourth aspect of the embodiments of the present application, an information data processing terminal is disclosed, which is used to implement the high-resolution extraction method of emotion patterns based on EEG signals.

[0059] The beneficial effects of the present invention are:

[0060] Traditional EEG signal analysis channels usually only focus on 4 or 5 classic frequency bands, the spectrum resolution is low, and the fine spectrum emotion pattern of EEG signals has not been explored. To address this problem, the present invention proposes a frequency domain refined EEG emotion pattern recognition method based on an emotion recognition model. The method constructs a fine frequency domain representation of multi-channel EEG signals based on FFT, and uses this as input to establish a recognition model, which can improve the emotion recognition performance.

[0061] The present invention proposes two strategies for serializing EEG signals. The two methods are grouping by the number of electrodes and grouping by the number of frequency domains. After being encoded by the encoder, the spatial representation and frequency domain representation of the EEG signals are obtained respectively. Considering the complementary nature of the spatial information and frequency domain information of EEG signals, the present invention concatenates these two feature representations for model pre-training, thereby enhancing the performance of emotion recognition.

[0062] The present invention divides the EEG information samples containing emotional information after FFT processing in the frequency domain, and then puts them into the encoder for processing, utilizing the global perception ability, multi-scale feature learning characteristics, model generalization ability, and scalability of the extraction model to improve the model's emotion recognition performance.

[0063] The present invention constructs two model grouping methods, namely grouping by the number of electrodes and grouping by the number of frequency domains. After being encoded by the encoder, due to the complementary nature between the time features and the spatial features, the obtained data is vectorized and spliced. Using this to train the classification model can effectively improve the accuracy of emotion recognition.

[0064] Finally, the present invention uses the integrated gradient to verify the physiological interpretability. After the gradient integral calculation is restored to the brain topography in the frequency domain, it can be observed that the strong related information generated by different emotions is located in different brain regions, proving that this method has physiological interpretability. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A composition diagram of the emotion pattern extraction model of the present invention;

[0066] Figure 2 This is a composition diagram of the spatial or frequency domain encoder of the present invention. DETAILED DESCRIPTION

[0067] In order to better understand the content of the present invention, an embodiment is given here.

[0068] Figure 1 A composition diagram of the emotion pattern extraction model of the present invention; Figure 2 This is a composition diagram of the spatial or frequency domain encoder of the present invention.

[0069] In a first aspect of the embodiments of the present application, a method for extracting high-resolution emotion patterns based on EEG signals is disclosed, comprising:

[0070] S1, obtaining an original EEG signal data set; the original EEG signal data set includes an EEG signal sequence and corresponding emotion label information;

[0071] S2, preprocessing the original EEG signal data set to obtain an EEG signal training data set;

[0072] S3, using the EEG signal training data set to train the emotion pattern extraction model to obtain a trained emotion pattern extraction model;

[0073] S4, using the trained emotion pattern extraction model to process the collected EEG signal to obtain an emotion pattern extraction result.

[0074] The emotion pattern extraction model includes: a preprocessing module, a frequency domain encoding module, a space encoding module and a classifier module;

[0075] The preprocessing module is used to preprocess the received EEG signal sequence to obtain a spatial signal set and a frequency domain signal set;

[0076] The frequency domain coding module is connected to the preprocessing module and the feature fusion module respectively, and is used to perform frequency domain coding processing on the received frequency domain signal set to obtain a frequency domain coded signal;

[0077] The spatial coding module is connected to the preprocessing module and the feature fusion module respectively, and is used to perform spatial coding processing on the received spatial signal set to obtain a spatially coded signal;

[0078] The classifier module is connected to the space coding module and the frequency domain coding module, and is used to classify the received frequency domain coding signal and space coding signal to obtain the emotion pattern recognition result.

[0079] The preprocessing of the original EEG signal data set to obtain an EEG signal training data set includes:

[0080] S21, performing data cleaning processing on the original EEG signal data set to obtain a cleaned data set;

[0081] S22, performing pattern detection processing on the cleaned data set to obtain an EEG signal training data set.

[0082] The performing pattern detection processing on the cleaned data set to obtain an EEG signal training data set includes:

[0083] S221, for each type of data attribute of the cleaned data set, using the data collection information of the data as an independent variable and the data value of the data as a dependent variable, perform autoregression-sliding average modeling to obtain regression models of the data attributes of each type;

[0084] S222, using the regression model, calculating and processing the independent variable to obtain a regression data value; determining whether the absolute value of the difference between the regression data value and the corresponding dependent variable value is greater than a set first regression discrimination threshold; if it is greater than the first regression discrimination threshold, deleting the data from the cleaned data set; if it is less than or equal to the first regression discrimination threshold, not processing the data;

[0085] S223, performing fusion processing on all data of the cleaned data set after executing S221 to S222 to obtain an EEG signal training data set.

[0086] The preprocessing module preprocesses the received EEG signal sequence to obtain a spatial signal set and a frequency domain signal set, including:

[0087] Performing spectrum conversion processing on the received EEG signal sequence of each electrode to obtain the EEG spectrum signal of each electrode;

[0088] The EEG spectrum signals of all electrodes are spliced ​​and processed to obtain EEG space-spectrum image information;

[0089] The EEG space-spectrum image information is processed in the frequency domain to obtain a frequency domain signal set; the frequency domain signal set X t1 The expression is

[0090] X t1 =[a1, a2, …, a F ],

[0091] Among them, a i is the frequency domain signal of the i-th frequency band, and F is the total number of frequency bands;

[0092] The EEG space-spectrum image information is spatially represented to obtain a spatial signal set; the spatial signal set X t2 The expression is

[0093]

[0094] Among them, b i is the frequency domain signal of the signal collected by the i-th electrode, T represents the transposition operation, and C is the total number of electrodes.

[0095] The frequency domain coding module includes an input module, a mask processing module, a position coding module, a frequency domain encoder, and a frequency domain decoder;

[0096] The mask processing module is connected to the input module and the position encoding module, and is used to perform mask processing on the signal sequence received by the input module according to a preset ratio value to obtain a masked sequence; the masked sequence has a corresponding signal sequence;

[0097] The position encoding module is connected to the frequency domain encoder and is used to perform position encoding processing on the masked sequence to obtain a first feature sequence;

[0098] The frequency domain encoder is connected to the frequency domain decoder and is used to encode the first feature sequence to obtain a frequency domain coded signal;

[0099] The frequency domain decoder is used to decode the frequency domain coded signal to obtain a frequency domain decoded signal;

[0100] The spatial encoding module includes an input module, a mask processing module, a position encoding module, a spatial encoder, and a spatial decoder;

[0101] The mask processing module is connected to the input module and the position encoding module, and is used to perform mask processing on the signal sequence received by the input module according to a preset ratio value to obtain a masked sequence;

[0102] The position encoding module is connected to the spatial encoder and is used to perform position encoding processing on the masked sequence to obtain a second feature sequence;

[0103] The spatial encoder is connected to the spatial decoder and is used to encode the second feature sequence to obtain a spatially coded signal;

[0104] The spatial decoder is used to decode the spatially encoded signal to obtain a spatially decoded signal;

[0105] The training process of the emotion pattern extraction model includes:

[0106] Pre-training the frequency domain coding module and the space coding module to obtain a pre-trained emotion pattern extraction model;

[0107] Initialize the number of training iterations;

[0108] Using the EEG signal sequence in the EEG signal training data set as input data, inputting it into the pre-trained emotion pattern extraction model to obtain an emotion prediction value;

[0109] Performing difference calculation processing on the emotion label information and the emotion prediction value of the input data to obtain a difference value; the loss function of the difference calculation processing is shown in formula (5);

[0110] Determine whether the difference value satisfies a convergence condition, and obtain a first determination result;

[0111] When the first judgment result is no, judging whether the training iteration number value is equal to the training number threshold, and obtaining a second judgment result;

[0112] When the second judgment result is no, determining that the model training state does not meet the training termination condition;

[0113] When the second judgment result is yes, determining that the model training state satisfies the training termination condition;

[0114] When the first judgment result is yes, determining that the model training state satisfies a training termination condition;

[0115] When the model training state does not meet the training termination condition, the classifier module is updated with the first parameter update model, the training iteration number is increased by 1, and the execution is triggered to use the EEG signal sequence in the EEG signal training data set as input data;

[0116] When the model training state satisfies the training termination condition, the training process of the pre-trained emotion pattern extraction model is completed to obtain a trained emotion pattern extraction model.

[0117] The pre-training process comprises:

[0118] Initialize the number of pre-training iterations;

[0119] Using the masked sequence as input data, and inputting it into the encoding module to be pre-trained;

[0120] Processing the input data using the encoding module to be pre-trained, and using the output of the decoder contained therein as a prediction signal sequence;

[0121] Performing difference calculation processing on the signal sequence corresponding to the masked sequence and the predicted signal sequence to obtain a difference value; the loss function of the difference calculation processing is shown in formula (4);

[0122] Determine whether the difference value satisfies a convergence condition, and obtain a third determination result;

[0123] When the third judgment result is no, judging whether the pre-training iteration count value is equal to the pre-training iteration count threshold, and obtaining a fourth judgment result;

[0124] When the fourth judgment result is no, determining that the model training state does not meet the training termination condition;

[0125] When the fourth judgment result is yes, determining that the model training state satisfies the training termination condition;

[0126] When the third judgment result is yes, determining that the model training state satisfies the training termination condition;

[0127] When the model training state does not meet the training termination condition, the second parameter update model is used to update the parameters of the encoding module to be pre-trained, the pre-training iteration number value is increased by 1, and the masked sequence is used as input data and input into the encoding module to be pre-trained.

[0128] When the model training state satisfies the training termination condition, the pre-training process is completed to obtain a pre-trained encoding module;

[0129] After the pre-training process of the frequency domain coding module and the space coding module is completed, a pre-trained emotion pattern extraction model is obtained.

[0130] The classification processing of the received coded signal is to obtain a classification result value after processing the received time-coded signal, and each classification result value has a corresponding emotion pattern extraction result. For example, when the classification result value is 1, the emotion pattern recognition result is happiness; when the classification result value is 2, the emotion pattern recognition result is sadness; when the classification result value is 3, the emotion pattern recognition result is disappointment; when the classification result value is 4, the emotion pattern recognition result is calm; when the classification result value is 5, the emotion pattern recognition result is excitement.

[0131] The convergence condition refers to that the difference value is less than a preset judgment threshold, and the value of the judgment threshold can be 20.

[0132] The pre-training times threshold is preset and may be 100.

[0133] The coding module to be pre-trained may be a frequency domain coding module or a spatial coding module.

[0134] The first parameter updating model is:

[0135]

[0136] θ←θ+v,

[0137] In the formula, x (i) is the ith EEG signal sequence in the EEG signal training data set, is the loss function, v is the parameter update value, θ is the parameter of the classifier module, η is the initial parameter learning rate, α is the momentum angle parameter, 0≤α≤1, It means to find the partial derivative of variable θ, f(x (i) ; θ) represents the emotion prediction value calculated by the pre-trained emotion pattern extraction model for the ith EEG signal sequence of the EEG signal training data set, f(·) is the calculation function corresponding to the pre-trained emotion pattern extraction model; exp represents the power operation of the constant e; η and α are preset values.

[0138] The second parameter update model is:

[0139]

[0140] θ←θ+v,

[0141] Where v is the parameter update value, θ is the parameter of a certain encoding module, η is the initial parameter learning rate, α is the momentum angle parameter, 0≤α≤1, It means to find partial derivatives with respect to variable θ. represents the difference value calculated between the i-th masked sequence and the predicted signal sequence, exp represents the power operation of the constant e; η and α are preset values.

[0142] The EEG spectrum signals of all electrodes are spliced ​​to obtain EEG space-spectrum image information, including:

[0143] The EEG spectrum signal of each electrode is taken as a column vector, and a matrix is ​​constructed using all column vectors. It is confirmed that the matrix is ​​the EEG space-spectrum image information.

[0144] The masking process is to randomly set some variables in the signal sequence to 0 according to a preset ratio.

[0145] The frequency domain encoder and the spatial encoder can be implemented by using the encoder module in Transformer.

[0146] The frequency domain decoder and the spatial decoder can be implemented by using the decoder module in Transformer.

[0147] The splicing process can be implemented by using the contact function.

[0148] The classifier module can be implemented using a support vector machine (SVM).

[0149] The values ​​of emotional patterns include happiness, sadness, disappointment, calmness, excitement, etc.

[0150] The data collection information of the data includes data collection time information and collection electrode sequence number information;

[0151] In a first aspect of the embodiment of the present application, a method for extracting high-resolution emotion patterns based on EEG signals is disclosed, and the formula thereof conforms to the table shown in Table 1. The method comprises:

[0152] Table 1 Formula symbols

[0153]

[0154] Step 1: First, in module 101, the EEG data is preprocessed and the EEG signal is converted into a space-spectrum image using FFT, represented as X∈R C×F , where C is the number of EEG electrodes and F is the resolution in the frequency domain. These images are serialized in the spatial and frequency domains and then put into the Transformer encoder for encoding, as shown in Figure 1 .

[0155] Step 1.1: First, collect EEG signals with emotion labels to form a data set. Perform appropriate preprocessing on all EEG signals, including noise removal, artifact correction, and filtering, to ensure data quality and signal accuracy.

[0156] Step 1.2: The EEG of each trial is divided into several non-overlapping 5-second windows. This can maintain the stationarity of the signal and ensure that the signal within the window is stable.

[0157] Step 1.3: Use FFT on these 5-second EEG windows to obtain the segmented fine spectrum representation of the EEG signal. Finally, these segmented representations are averaged across trials, and the averaged fine spectrum is represented as X∈R C×F , where C is the number of EEG electrodes and F is the frequency domain resolution, see module 101.

[0158] Step 1.4: Spatial serialization. The spatial serialization method divides the segmented fine spectral representation of the EEG signal into multiple spatial sequences, each sequence contains all electrodes. These sequences are then used as the sequence input of an EEG trial, see module 103. This design utilizes the spatial information in the EEG signal. The formula of the spatial serialization method is:

[0159] X=[a1,a2,…,a F ] (1)

[0160] where a i ∈R C , C is the number of EEG signal electrodes. The spatial representation is calculated by spatial Transformer as follows:

[0161]

[0162] Step 1.5: Spectral serialization. In addition to spatial information, spectral information is another important dimension to represent EEG signals. In order to enable the emotion pattern extraction model to effectively capture the global emotion characteristics in the frequency domain, the electrode-by-electrode serialization method is adopted, see module 102. Each block contains complete frequency domain information. The formula of the spectral serialization method is:

[0163]

[0164] where b i ∈R F , T is the transpose operation, and F is the frequency domain resolution. The spectrum representation is calculated by the spectrum Transformer as follows:

[0165]

[0166] Step 2: After position encoding, the sequence data is put into the Transformer encoder to encode the features for emotion classification; the two encoders are pre-trained using a mask-based self-supervised learning method to improve the classification accuracy, such as Figure 1 .

[0167] Step 2.1: Taking module 102 as an example, firstly, the grouped serialized data is normalized and then put into the multi-head attention mechanism in the encoder. The operation process of the encoder is as follows: Figure 2 Module 106 in.

[0168] Step 2.2: The self-attention mechanism is used in the multi-head attention mechanism. First, the transmitted data is put into the fully connected layer to obtain three new matrices of the same dimension: query Q (Query), key K (Key), value V (Value).

[0169] Step 2.3: Split the three matrices Q, K, and V to implement the multi-head attention mechanism. Suppose the dimension of the original data is d, the number of repetitions of the original attention mechanism is h, and the dimensions of the three matrices Q, K, and V after segmentation are d q d k d v , then d q =d k =d v =d / h.

[0170] Step 2.4: Each head follows the formula Perform operations and concatenate the results of each head operation to obtain the final output result.

[0171] Step 2.5: Add the result to the data before input and normalize it. The purpose is to standardize the data, which can improve the performance and stability of the subsequent model.

[0172] Step 2.6: Put the normalized data into a feedforward neural network, which is a two-layer fully connected layer with a ReLU activation function in between, and add the result obtained with the result of step 2.5 and normalize, which is also a normalization operation. Module 103 can be obtained in the same way.

[0173] Step 2.7: In modules 102 and 103, two encoders are pre-trained using a mask-based self-supervised learning method to improve classification accuracy. Take the spectrum branch as an example (see Figure 1), the segmented fine spectral representation of the EEG signal is divided into multiple blocks in the frequency domain to form a sequence. On this sequence, a certain proportion of the sequence is masked using a random selection process. The visible sequence is then used to predict the masked sequence. The loss of the difference between the predicted sequence and the masked sequence is calculated. The loss function of self-supervised learning consists of two parts, as described below:

[0174]

[0175] Step 3: The two types of sequences obtained after encoding are concatenated to obtain features, which are used for emotion extraction in module 104 (the emotion classifier is implemented using the SVM module), and finally the emotion pattern extraction result is obtained. The loss function of classification training is as follows:

[0176]

[0177] Where y is the true label, is the predicted label, and N is the number of samples. i represents the label of the i-th category in the actual value, Represents the label of the i-th category in the predicted value.

[0178] Step 4: After obtaining the emotion classification result, the result can be used to perform integral gradient calculation in module 105, such as Figure 1 As shown. The Integral Gradient (IG) method is used to explain the patterns learned by the proposed model. The basic idea of ​​IG is to calculate the gradient in the input space and quantify the cumulative effect of the gradient by integration. For each test sample, the maximum component is selected using the index of its true label to calculate the attribution result based on IG. Subsequently, these attribution results are grouped according to the true label and averaged to obtain the attribution result for each emotion. The calculation formula is as follows, let the function mapping be represented by F, and the integral path function be γ i (α), α represents. The attribution results obtained by gradient integral calculation are restored to brain topography in the frequency domain. It can be observed that the strong related information generated by different emotions is located in different brain regions, proving that this method has physiological interpretability.

[0179] In a second aspect of the embodiment of the present application, a device for extracting high-resolution emotion patterns based on EEG signals is disclosed, the device comprising:

[0180] A memory storing executable program code;

[0181] a processor coupled to the memory;

[0182] The processor calls the executable program code stored in the memory to execute the method for high-resolution extraction of emotion patterns based on EEG signals.

[0183] In a third aspect of an embodiment of the present application, a computer-storable medium is disclosed, wherein the computer-storable medium stores computer instructions, and when the computer instructions are called by a computer, they are used to execute the method for high-resolution extraction of emotion patterns based on EEG signals.

[0184] In a fourth aspect of the embodiments of the present application, an information data processing terminal is disclosed, which is used to implement the high-resolution extraction method of emotion patterns based on EEG signals.

[0185] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for extracting emotional patterns with high resolution based on EEG signals, characterized in that: include: S1, obtain the original data set of EEG signals; The EEG signal raw data set includes EEG signal sequences and corresponding emotion label information; S2, preprocessing the original EEG signal data set to obtain an EEG signal training data set; S3, using the EEG signal training data set to train the emotion pattern extraction model to obtain a trained emotion pattern extraction model; S4, using the trained emotion pattern extraction model to process the collected EEG signal to obtain an emotion pattern extraction result.

2. The method for extracting emotion patterns based on EEG signals with high resolution according to claim 1, characterized in that: The emotion pattern extraction model includes: a preprocessing module, a frequency domain encoding module, a space encoding module and a classifier module; The preprocessing module is used to preprocess the received EEG signal sequence to obtain a spatial signal set and a frequency domain signal set; The frequency domain coding module is connected to the preprocessing module and the feature fusion module respectively, and is used to perform frequency domain coding processing on the received frequency domain signal set to obtain a frequency domain coded signal; The spatial coding module is connected to the preprocessing module and the feature fusion module respectively, and is used to perform spatial coding processing on the received spatial signal set to obtain a spatially coded signal; The classifier module is connected to the space coding module and the frequency domain coding module, and is used to classify the received frequency domain coding signal and space coding signal to obtain the emotion pattern recognition result.

3. The method for extracting emotion patterns based on EEG signals with high resolution according to claim 1, characterized in that: The preprocessing of the original EEG signal data set to obtain an EEG signal training data set includes: S21, performing data cleaning processing on the original EEG signal data set to obtain a cleaned data set; S22, performing pattern detection processing on the cleaned data set to obtain an EEG signal training data set.

4. The method for extracting emotion patterns with high resolution based on EEG signals as claimed in claim 3, characterized in that: The performing pattern detection processing on the cleaned data set to obtain an EEG signal training data set includes: S221, for each type of data attribute of the cleaned data set, using the data collection information of the data as an independent variable and the data value of the data as a dependent variable, perform autoregression-sliding average modeling to obtain regression models of the data attributes of each type; S222, using the regression model, calculating and processing the independent variable to obtain a regression data value; determining whether the absolute value of the difference between the regression data value and the corresponding dependent variable value is greater than a set first regression discrimination threshold; if it is greater than the first regression discrimination threshold, deleting the data from the cleaned data set; if it is less than or equal to the first regression discrimination threshold, not processing the data; S223, performing fusion processing on all data of the cleaned data set after executing S221 to S222 to obtain an EEG signal training data set.

5. The method for extracting emotional patterns with high resolution based on EEG signals as claimed in claim 2, characterized in that: The preprocessing module preprocesses the received EEG signal sequence to obtain a spatial signal set and a frequency domain signal set, including: Performing spectrum conversion processing on the received EEG signal sequence of each electrode to obtain the EEG spectrum signal of each electrode; The EEG spectrum signals of all electrodes are spliced ​​and processed to obtain EEG space-spectrum image information; The EEG space-spectrum image information is processed in the frequency domain to obtain a frequency domain signal set; the frequency domain signal set X t1 The expression is X t1 =[a1,a2,…,a F ], Among them, a i is the frequency domain signal of the i-th frequency band, and F is the total number of frequency bands; The EEG space-spectrum image information is spatially represented to obtain a spatial signal set; the spatial signal set X t2 The expression is Among them, b i is the frequency domain signal of the signal collected by the i-th electrode, T represents the transposition operation, and C is the total number of electrodes.

6. The method for extracting emotional patterns with high resolution based on EEG signals as claimed in claim 5, characterized in that: The frequency domain coding module includes an input module, a mask processing module, a position coding module, a frequency domain encoder, and a frequency domain decoder; The mask processing module is connected to the input module and the position encoding module, and is used to perform mask processing on the signal sequence received by the input module according to a preset ratio value to obtain a masked sequence; The masked sequence has a corresponding signal sequence; The position encoding module is connected to the frequency domain encoder and is used to perform position encoding processing on the masked sequence to obtain a first feature sequence; The frequency domain encoder is connected to the frequency domain decoder and is used to encode the first feature sequence to obtain a frequency domain coded signal; The frequency domain decoder is used to decode the frequency domain coded signal to obtain a frequency domain decoded signal; The spatial encoding module includes an input module, a mask processing module, a position encoding module, a spatial encoder, and a spatial decoder; The mask processing module is connected to the input module and the position encoding module, and is used to perform mask processing on the signal sequence received by the input module according to a preset ratio value to obtain a masked sequence; The position encoding module is connected to the spatial encoder and is used to perform position encoding processing on the masked sequence to obtain a second feature sequence; The spatial encoder is connected to the spatial decoder and is used to encode the second feature sequence to obtain a spatially coded signal; The spatial decoder is used to decode the spatially encoded signal to obtain a spatially decoded signal.

7. The method for extracting emotional patterns with high resolution based on EEG signals as claimed in claim 5, characterized in that: The training process of the emotion pattern extraction model includes: Pre-training the frequency domain coding module and the space coding module to obtain a pre-trained emotion pattern extraction model; Initialize the number of training iterations; Using the EEG signal sequence in the EEG signal training data set as input data, inputting it into the pre-trained emotion pattern extraction model to obtain an emotion prediction value; Performing difference calculation processing on the emotion label information and the emotion prediction value of the input data to obtain a difference value; Determine whether the difference value satisfies a convergence condition, and obtain a first determination result; When the first judgment result is no, judging whether the training iteration number value is equal to the training number threshold, and obtaining a second judgment result; When the second judgment result is no, determining that the model training state does not meet the training termination condition; When the second judgment result is yes, determining that the model training state satisfies the training termination condition; When the first judgment result is yes, determining that the model training state satisfies a training termination condition; When the model training state does not meet the training termination condition, the classifier module is updated with the first parameter update model, the training iteration number is increased by 1, and the execution is triggered to use the EEG signal sequence in the EEG signal training data set as input data; When the model training state satisfies the training termination condition, the training process of the pre-trained emotion pattern extraction model is completed to obtain a trained emotion pattern extraction model.

8. A high-resolution extraction device for emotion patterns based on EEG signals, characterized in that: The device comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the high-resolution extraction method of emotion patterns based on EEG signals as described in any one of claims 1 to 7.

9. A computer storable medium, characterized in that: The computer storable medium stores computer instructions, which, when called by a computer, are used to execute the method for high-resolution extraction of emotion patterns based on EEG signals as claimed in any one of claims 1 to 7.

10. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the high-resolution extraction method of emotion patterns based on EEG signals as described in any one of claims 1 to 7.

Images