Emotional feature classification method for EEG signals based on transfer learning

Abstract

The present invention belongs to the field of medical artificial intelligence and emotional brain-computer interaction, and specifically relates to a method, system, and device for classifying emotional features of EEG signals based on transfer learning, which aims to solve the problem that the existing EEG emotional feature classification method cannot overcome the poor accuracy of cross-individual EEG emotion classification caused by the inconsistent distribution of EEG data of different individuals. The method includes: obtaining emotional EEG signal data to be classified as input data; extracting the differential entropy features of the input data, and inputting the differential entropy features into the multi-layer perceptron of the EEG signal emotion classification model to obtain deep features; based on the deep features, obtaining the classification results corresponding to the input data through the classifier of the EEG signal emotion classification model. The present invention not only overcomes the problem of inconsistent distribution of EEG data of different individuals, but also has a good classification effect in cross-individual EEG emotion classification, while also ensuring the necessary generalization and robustness.

Description

Technical Field

[0001] The present invention belongs to the fields of medical artificial intelligence and emotional brain-computer interaction, and specifically relates to a method, system, and device for classifying emotional features of EEG signals based on transfer learning. Background Art

[0002] Among the many emotion recognition methods, EEG (Electroencephalogram) data has obvious advantages in terms of reliability and accuracy. However, individual differences in EEG limit the generalization ability of emotion classifiers between subjects. In addition, due to the non-stationary nature of EEG, the signals of subjects will change over time, which is a major challenge for emotion recognition across time. As can be seen from the above content, unlike other mainstream classification problems, in individual independent problems based on EEG data, the EEG data of different people are distributed differently, resulting in poor results of conventional methods. An important premise of conventional machine learning is that existing data and unknown data obey the same laws, that is, statistically speaking, they obey the same distribution. How to narrow the distribution of EEG data of different people is the key to solving the problem. Therefore, the method of transfer learning is introduced to complete the classification of emotional EEG signals.

[0003] Based on this, the present invention combines global-local adversarial and joint domain adaptation for the first time to realize a transfer learning-based method for classifying emotional features of EEG signals. Summary of the Invention

[0004] In order to solve the above-mentioned problems in the prior art, that is, to solve the problem that the existing EEG emotion feature classification method cannot overcome the problem of poor accuracy in cross-individual EEG emotion classification caused by inconsistent distribution of EEG data of different individuals, the first aspect of the present invention proposes a method for classifying EEG signal emotion features based on transfer learning, which includes:

[0005] S100, obtaining emotional EEG signal data to be classified as input data;

[0006] S200, extracting differential entropy features of the input data, and inputting the differential entropy features into a multi-layer perceptron of an EEG signal emotion classification model to obtain depth features corresponding to the input data;

[0007] S300, obtaining a classification result corresponding to the input data through a classifier of an EEG signal emotion classification model based on the deep features;

[0008] Among them, the EEG signal emotion classification model includes a multi-layer perceptron and a classifier.

[0009] In some preferred embodiments, the EEG signal emotion classification model is trained by:

[0010] A100, obtain a first data set and a second data set; the first data set is a source domain training data set, which includes training samples and their corresponding true classification labels; the second data set is a target domain training data set; the training samples are emotional EEG signal data;

[0011] A200, extract the differential entropy features D of each training sample in the first training set respectively s And the differential entropy feature D of each training sample in the second training set t After extraction, the differential entropy features are input into the multi-layer perceptron of the EEG signal emotion classification model to obtain the deep features f of each training sample in the first training set. s , the deep features f of each training sample in the second training set t ;

[0012] A300, will f s 、f t Input the pre-built global domain adversarial module and calculate the global domain adversarial loss; s 、f t Input the softmax of the classifier, and then input the pre-built local domain adversarial module to calculate the local domain adversarial loss;

[0013] A400, combined with the preset balance coefficient, balances the global domain adversarial loss and the local domain adversarial loss to obtain the global-local domain adversarial loss;

[0014] A500, maps Ds and Dt to a low-dimensional space through a fully connected layer, denoted as A i and B j ; Calculate A i 、B j The similarity between them is used to calculate the probability of transferring from any element of the first matrix to any element of the second matrix. , the probability of transferring from any element of the second matrix to any element of the first matrix Then we get the probability of transferring from any element of the first matrix to any element of the second matrix and then to any element of the first matrix. ;based on Calculate the measurement loss of the similarity between the same category label data and the measurement loss of the similarity between the source domain and the target domain;

[0015] Among them, A represents the feature matrix after the source domain training sample is mapped to the low-dimensional space, which is used as the first matrix, and B represents the feature matrix after the target domain training sample is mapped to the low-dimensional space, which is used as the second matrix;

[0016] A600, which is a weighted sum of the global-local domain adversarial loss, the loss of similarity between labeled data of the same category, and the loss of similarity between the source and target domains, to obtain the joint domain adaptation loss;

[0017] A700 inputs the deep features into the classifier to obtain a classification result, which is used as a prediction result; based on the prediction result and the true classification label, calculates the classification loss using a cross-entropy loss function; sums the classification loss with the joint domain adaptation loss to obtain a total loss, and then updates the network parameters of the EEG signal emotion classification model;

[0018] A800, loop steps A100-A700 until a trained EEG signal emotion classification model is obtained.

[0019] In some preferred embodiments, the global domain adversarial module and the local domain adversarial module are both discriminators constructed with multiple fully connected layers.

[0020] In some preferred embodiments, the global-local domain adversarial loss is calculated as follows:

[0021] L doamin =(1-ω)L g +ωL l

[0022]

[0023]

[0024] Among them, L doamin represents the global-local domain adversarial loss, L g represents the global domain adversarial loss, L l represents the sum of local domain adversarial losses, n s 、n t Respectively represent the number of training samples of the source domain training dataset and the target domain training dataset, x i represents the training sample, d i represents the domain label corresponding to each domain, G d Indicates gradient reversal, G f Represents the feature extractor, that is, the multi-layer perceptron, L d represents the cross entropy loss function, ω represents the balance coefficient, i represents the subscript, represents f s or f t The probability obtained after softmax processing, C represents the set of category labels, c represents a specific category label, represents the gradient reversal feature of the local domain discriminator corresponding to category c, The local domain discriminator corresponding to the c category is based on the gradient reversal feature and the label The resulting cross entropy loss.

[0025] In some preferred embodiments, the preset balance coefficient is:

[0026]

[0027]

[0028] in, represents the local domain adversarial loss.

[0029] In some preferred embodiments, the joint domain adaptation loss is calculated as follows:

[0030] L trans =L doamin +β1L walker +β2L visit

[0031] L walker =H(T,P aba )

[0032]

[0033] L visit =(V, P visit )

[0034]

[0035]

[0036]

[0037]

[0038] Among them, L trans represents the joint domain adaptation loss, L walker Indicates the measurement loss of similarity between data with the same category label, L visit represents the loss of similarity between the source domain and the target domain, β1 and β2 represent the preset regularization coefficients, H represents the cross entropy loss function, and M ij Indicates A i 、B j The similarity between them, class() represents the category, T represents P aba The label of the transition probability of the same type of elements in the case of uniform distribution, y represents P ab The transition probabilities follow uniformly distributed labels.

[0039] In some preferred embodiments, the total loss is calculated as follows:

[0040] L=L trans +L y

[0041] L y =H(y, A)

[0042] Among them, L represents the total loss, L y Represents the classification loss, y represents the true classification label, and A represents the feature matrix after the source domain training sample is mapped to the low-dimensional space.

[0043] The second aspect of the present invention proposes a system for classifying emotion features of EEG signals based on transfer learning, comprising: a data acquisition module, a feature extraction module, and an emotion classification module;

[0044] The data acquisition module is configured to acquire emotional EEG signal data to be classified as input data;

[0045] The feature extraction module is configured to extract the differential entropy features of the input data and input the differential entropy features into the multi-layer perceptron of the EEG signal emotion classification model to obtain the deep features corresponding to the input data;

[0046] The emotion classification module is configured to obtain a classification result corresponding to the input data based on the deep features through a classifier of the EEG signal emotion classification model;

[0047] Among them, the EEG signal emotion classification model includes a multi-layer perceptron and a classifier.

[0048] The third aspect of the present invention proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the above-mentioned method for classifying emotional features of EEG signals based on transfer learning.

[0049] In a fourth aspect of the present invention, a computer-readable storage medium is proposed, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-mentioned method for classifying emotional features of EEG signals based on transfer learning.

[0050] Beneficial effects of the present invention:

[0051] The present invention not only overcomes the problem of inconsistent distribution of EEG data of different individuals, but also has a good classification effect in cross-individual EEG emotion classification, while also ensuring the necessary generalization and robustness.

[0052] 1) The proposed method has a simple network structure, requires a small number of weights, and eliminates the need to build custom feature extractors for different individuals. Experiments using the proposed method on a cross-temporal EEG emotion classification task achieved an accuracy of 89.4%, demonstrating its excellent classification performance across a variety of emotion recognition tasks.

[0053] 2) At the EEG signal feature level, the method of the present invention takes into account the global and local differences of EEG features of different individuals, fully utilizes the domain adversarial method, and combines it with the joint domain adaptation method to balance it to form a transfer learning module, which can better make the EEG signal features of different people tend to be uniformly distributed and distinguish the emotion category features from each other, which can effectively improve the problem of different EEG signal features of different people being in different distributions. The final result has good generalization performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Other features, objects and advantages of the present application will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings.

[0055] Figure 1 This is a flow chart of a method for classifying emotional features of EEG signals based on transfer learning according to an embodiment of the present invention;

[0056] Figure 2 This is a schematic diagram of a framework of a system for classifying emotional features of EEG signals based on transfer learning according to an embodiment of the present invention;

[0057] Figure 3 1 is a simplified schematic diagram of the training process of the EEG signal emotion classification model according to an embodiment of the present invention;

[0058] Figure 4 Detailed schematic diagram of the training process of the EEG signal emotion classification model according to one embodiment of the present invention;

[0059] Figure 5 It is a structural diagram of a computer system suitable for implementing an electronic device of an embodiment of the present application according to an embodiment of the present invention. DETAILED DESCRIPTION

[0060] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0061] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0062] It should be noted that, unless there is any conflict, the embodiments and features in the embodiments of this application can be combined with each other.

[0063] The present invention is based on the transfer learning method for the classification of EEG signal emotion features, such as Figure 1 As shown, the following steps are included:

[0064] S100, obtaining emotional EEG signal data to be classified as input data;

[0065] S200, extracting differential entropy features of the input data, and inputting the differential entropy features into a multi-layer perceptron of an EEG signal emotion classification model to obtain depth features corresponding to the input data;

[0066] S300, obtaining a classification result corresponding to the input data through a classifier of an EEG signal emotion classification model based on the deep features;

[0067] Among them, the EEG signal emotion classification model includes a multi-layer perceptron and a classifier.

[0068] In order to more clearly illustrate the EEG signal emotion feature classification system based on transfer learning of the present invention, each step in an embodiment of the method of the present invention is described in detail below with reference to the accompanying drawings.

[0069] In the following embodiments, the training process of the EEG signal emotion classification model is first described in detail, and then the process of obtaining the classification results of the emotion EEG signal data based on the EEG signal emotion feature classification method based on transfer learning is described in detail.

[0070] 1. Training process of EEG signal emotion classification model

[0071] The present invention extracts differential entropy (DE) features from emotional EEG signal data, inputs them into a multi-layer perceptron, and outputs deep features. Using deep features for transfer learning mainly involves inputting deep features into the global-local domain adversarial module and the joint domain adaptation module to obtain transfer learning loss. The deep features are input into the classifier module to calculate the cross entropy to obtain the classification loss. The sum of the obtained transfer loss and classification loss is used for error back propagation, as shown in Figure 2. Figure 3 The details are as follows:

[0072] A100, obtain a first data set and a second data set; the first data set is a source domain training data set, which includes training samples and their corresponding true classification labels; the second data set is a target domain training data set; the training samples are emotional EEG signal data;

[0073] A200, extract the differential entropy features D of each training sample in the first training set respectively s And the differential entropy feature D of each training sample in the second training set t After extraction, the differential entropy features are input into the multi-layer perceptron of the EEG signal emotion classification model to obtain the deep features f of each training sample in the first training set. s , the deep features f of each training sample in the second training set t ;

[0074] In this embodiment, there are two data sets. One data set includes ns training samples from the source domain and their corresponding true classification labels y (such as positive, negative), which is called the source domain training data set (i.e., training set). The other data set includes n t training samples from the target domain (i.e., validation set), where the training sample represents x i , the domain label corresponding to each domain represents d i (belonging to the source domain or target domain, such as Figure 4 in Both represent domain labels, and c is used to distinguish categories).

[0075] Extract the differential entropy features (DE features) D of each training sample in the source domain training dataset respectively s And the differential entropy feature D of each training sample in the target domain training dataset t After extraction, the differential entropy features are input into the multi-layer perceptron of the EEG signal emotion classification model to obtain the deep features f of each training sample in the first training set. s , the deep features f of each training sample in the second training set t ,like Figure 4 shown.

[0076] A300, will f s 、f t Input the pre-built global domain adversarial module and calculate the global domain adversarial loss; s 、f t Input the softmax of the classifier, and then input the pre-built local domain adversarial module to calculate the local domain adversarial loss;

[0077] In this embodiment, fine-grained adversarial analysis of deep features of EEG signal source and target domains is implemented to make EEG signal features in different domains tend to be uniformly distributed, and the adversarial loss obtained is used to calculate the transfer loss. Specifically:

[0078] First, the deep features are input into the global domain adversarial module to obtain the global domain adversarial loss. The global domain adversarial module is a discriminator (or discriminator for short) constructed by multiple fully connected layers, so the global domain adversarial module is also called the global domain discriminator module, such as Figure 4 shown).

[0079] Global domain adversarial loss L g , which is calculated as follows:

[0080]

[0081] Among them, G f Represents the feature extractor, that is, the multi-layer perceptron, L d (G d (G f (x i )),d i ) indicates that gradient reversal G is used for the feature d The method computes domain adversarial cross entropy loss, L d represents the cross entropy loss, G d Indicates gradient reversal.

[0082] Next, the deep features are simultaneously input into the softmax of the classifier, and then input into the local domain adversary (i.e., the local domain adversarial module, which is a discriminator built by multiple fully connected layers, and the local domain adversarial module is also called the local domain discriminator module, such as Figure 4 As shown), the losses of these several local domain adversaries are summed to obtain the local domain adversarial loss and the sum of local domain adversarial losses L l , as shown in the following formula:

[0083]

[0084]

[0085] in, represents f s or f t The probability obtained after softmax processing belongs to [0, 1]. Multiplication reflects the scaling, that is, Figure 4 in The global domain adversarial loss formula does not have this term because the global domain discriminator does not go through softmax or the probability is always 1. represents the gradient reversal feature of the local domain discriminator corresponding to category c, The local domain discriminator representing the c category is based on the gradient reversal feature and the label The resulting cross entropy loss, C represents the set of category labels, and c represents a specific category label.

[0086] A400, combined with the preset balance coefficient, balances the global domain adversarial loss and the local domain adversarial loss to obtain the global-local domain adversarial loss;

[0087] In this embodiment, the global domain adversarial loss and the local domain adversarial loss are balanced by formula (4), and the global-local domain adversarial loss L is obtained. doamin (Right now Figure 4 Loss in d ):

[0088] L doamin =(1-ω)L g +ωL l (4)

[0089] Where ω is the balance coefficient, which is calculated by formula (5):

[0090]

[0091] A500, maps Ds and Dt to a low-dimensional space through a fully connected layer, denoted as A i and B j ; Calculate A i 、B j The similarity between them is used to calculate the probability of transferring from any element of the first matrix to any element of the second matrix. The probability of transferring from any element of the second matrix to any element of the first matrix Then we get the probability of transferring from any element of the first matrix to any element of the second matrix and then to any element of the first matrix. based on Calculate the measurement loss of the similarity between the same category label data and the measurement loss of the similarity between the source domain and the target domain;

[0092] Although EEG signals from different individuals reside in different domains, their inherent properties (emotional categories) are the same, so it is a reasonable assumption to assume that their features are similar. Based on this, data from the source and target domains can be mapped into a low-dimensional space to extract the similarity between samples from different domains.

[0093] In this embodiment, a joint domain adaptation method is used to measure the expected distribution of source and target domain data, and the obtained loss is used to calculate the migration loss. Specifically:

[0094] For the joint domain adaptation module for EEG signal source domain and target domain EEG emotion features, Ds and D t Mapped to low-dimensional space, denoted as A i and B j , and its corresponding feature sample is recorded as and Through the inner product M ij To identify the source domain A i and target domain B j The similarity of the sample and The association probability, that is, the probability of transferring from any element of the first matrix to any element of the second matrix, can be expressed as follows:

[0095]

[0096] Among them, A represents the feature matrix after the source domain training sample is mapped to the low-dimensional space, which is used as the first matrix with a dimension of n s *d, B represents the feature matrix after the target domain training sample is mapped to the low-dimensional space, as the second matrix, the dimension is n t *d, d represents the dimension of the feature of the current training sample; It represents the similarity between element i in A and element j in B. Similarity itself does not have a direction, but it can be understood here as the probability of transition from i to j.

[0097] Similarly, the probability of the target domain sample being associated with the source domain sample is defined as the probability of transferring from any element of the second matrix to any element of the first matrix: It can be understood as the probability of transition from j->i (actually the covariance matrix transpose of ).

[0098] The present invention expects that the labels remain unchanged during the transfer process, for example: positive, neutral, and negative emotions. The target domain does not have category labels at this time, but the source domain data does. This achieves the measurement of the similarity between the source and target domains, while utilizing the category labels of the source domain. The result of the expected transfer is a uniform distribution, that is, the probability of each element in the source domain A after the transfer from A->B->A is the same, and T represents P aba The transition probability of the same type of elements in the label is in accordance with the uniform distribution, that is, the same class is as similar as possible, and different classes are not similar, so as to achieve the data distribution within the same class closer. That is, the cycle P of the joint domain adaptation expected association measure aba It should be a uniform distribution T, which can be calculated by the cross entropy H, as shown in formula (8):

[0099]

[0100] L walker=H(T,P aba ) (8)

[0101]

[0102] Among them, L walker It represents the measurement loss of the similarity between the labeled data of the same category.

[0103] In addition, in order to discover more similarities between the source domain and the target domain, L visit To make the two domains cover as much of the target domain as possible when performing association measurement.

[0104] L visit =(V, P visit ) (10)

[0105]

[0106] Among them, L visit It represents the loss of similarity between the source domain and the target domain, V represents P ab The transition probabilities follow uniformly distributed labels.

[0107] A600, which is a weighted sum of the global-local domain adversarial loss, the loss of similarity between labeled data of the same category, and the loss of similarity between the source and target domains, to obtain the joint domain adaptation loss;

[0108] In this embodiment, by walker 、L visit The two are combined and balanced to achieve the goal of bringing the source domain and the target domain closer and maintaining high similarity between similar samples, that is, the loss of the joint domain adaptation part (joint domain adaptation loss L trans , which can also be called migration loss, such as Figure 3 As shown), it can be expressed as formula (11), where β is a fixed regularization term.

[0109] L tranns =L doamin +β1L walker +β2L visit (12)

[0110] Among them, β1 and β2 represent the preset regularization coefficients, L trans Right now Figure 4 Loss in a .

[0111] A700 inputs the deep features into the classifier to obtain a classification result, which is used as a prediction result; based on the prediction result and the true classification label, calculates the classification loss using a cross-entropy loss function; sums the classification loss with the joint domain adaptation loss to obtain a total loss, and then updates the network parameters of the EEG signal emotion classification model;

[0112] In this embodiment, the classifier is a fully connected layer classifier, including a fully connected layer (i.e. Figure 4 f in ), classifier (such as softmax).

[0113] The total loss L in the transfer learning part trans After the calculation is completed, the classification loss L is calculated for the classifier module based on the deep feature, combined with the category label y of the source domain. y (Right now Figure 4 Loss in y ), and then the overall loss (i.e., total loss) L of the entire network is obtained:

[0114] L y =H(y, A) (13)

[0115] L=L trans +L y (14)

[0116] A800, loop steps A100-A700 until a trained EEG signal emotion classification model is obtained.

[0117] In this embodiment, the network is back-propagated and optimized using the total loss L, and a trained EEG signal emotion classification model is finally iterated.

[0118] 2. Emotional feature classification method for EEG signals based on transfer learning

[0119] S100, obtaining emotional EEG signal data to be classified as input data;

[0120] In this embodiment, the emotional EEG signal data to be classified is first obtained.

[0121] S200, extracting differential entropy features of the input data, and inputting the differential entropy features into a multi-layer perceptron of an EEG signal emotion classification model to obtain depth features corresponding to the input data;

[0122] S300 , based on the deep features, obtain a classification result corresponding to the input data through a classifier of an EEG signal emotion classification model.

[0123] In this embodiment, the differential entropy features of the emotional EEG signal data are extracted, and the classification results corresponding to the emotional EEG signal data are obtained through the multi-layer perceptron and classifier of the EEG signal emotion classification model.

[0124] This paper combines global-local adversarial learning and joint domain adaptation for the first time to implement a transfer learning-based method for classifying emotional features of EEG signals. A deep neural network is used to extract emotional features from EEG data. Taking into account individual differences, an adversarial approach is proposed to narrow the distance between EEG data domains between subjects. To achieve better results, both global and local adversarial learning between domains is considered, and a balance factor is used to adjust the ratio of the two to approximately adjust the joint distribution. This addresses the challenge of EEG non-stationarity, where the subject's signal changes over time.

[0125] The second embodiment of the present invention is a system for classifying emotional features of EEG signals based on transfer learning, such as Figure 2 As shown, it includes: a data acquisition module 100, a feature extraction module 200, and a sentiment classification module 300;

[0126] The data acquisition module 100 is configured to acquire emotional EEG signal data to be classified as input data;

[0127] The feature extraction module 200 is configured to extract the differential entropy features of the input data and input the differential entropy features into the multi-layer perceptron of the EEG signal emotion classification model to obtain the depth features corresponding to the input data;

[0128] The emotion classification module 300 is configured to obtain a classification result corresponding to the input data based on the deep features through a classifier of the EEG signal emotion classification model;

[0129] Among them, the EEG signal emotion classification model includes a multi-layer perceptron and a classifier.

[0130] It should be noted that the EEG signal emotion feature classification system based on transfer learning provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be decomposed or combined. For example, the modules in the above embodiments can be combined into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps and are not regarded as improper limitations on the present invention.

[0131] An electronic device according to the third embodiment of the present invention comprises: at least one processor; and a memory communicatively connected to at least one of the processors; wherein the memory stores instructions executable by the processor, and the instructions are used to be executed by the processor to implement the above-mentioned method for classifying emotional features of EEG signals based on transfer learning.

[0132] A fourth embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-mentioned method for classifying emotional features of EEG signals based on transfer learning.

[0133] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes and related instructions of the storage device and processing device described above can refer to the corresponding processes in the aforementioned method examples and will not be repeated here.

[0134] Reference below Figure 5 , which shows a structural diagram of a server computer system suitable for implementing the method, system, and device embodiments of the present application. Figure 5 The server shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0135] like Figure 5 As shown, the computer system includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage part 508 into the random access memory (RAM) 503. Various programs and data required for system operation are also stored in the RAM 503. The CPU 501, ROM 502 and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0136] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, and the like; an output section 507 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 510 as needed so that a computer program read therefrom can be installed into the storage section 508 as needed.

[0137] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication portion 509 and / or installed from a removable medium 511. When the computer program is executed by the central processing unit (CPU501), the above functions defined in the method of the present application are performed. It should be noted that the computer-readable medium mentioned above in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0138] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0139] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0140] The terms "first", "second", etc. are used to distinguish similar objects, rather than to describe or indicate a particular order or sequence.

[0141] The term "comprise" or any other similar term is intended to cover non-exclusive inclusion such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0142] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

Claims

1. A method for classifying emotional features of EEG signals based on transfer learning, characterized in that: The method comprises the following steps: S100, obtaining emotional EEG signal data to be classified as input data; S200, extracting differential entropy features of the input data, and inputting the differential entropy features into a multi-layer perceptron of an EEG signal emotion classification model to obtain depth features corresponding to the input data; S300, obtaining a classification result corresponding to the input data through a classifier of an EEG signal emotion classification model based on the deep features; Wherein, the EEG signal emotion classification model includes a multi-layer perceptron and a classifier; The training method of the EEG signal emotion classification model is as follows: A100, obtain a first data set and a second data set; the first data set is a source domain training data set, which includes training samples and their corresponding true classification labels; the second data set is a target domain training data set; the training samples are emotional EEG signal data; A200 extracts differential entropy features Ds of each training sample in the first dataset and differential entropy features Dt of each training sample in the second dataset. After extraction, the differential entropy features are input into a multi-layer perceptron of an EEG signal emotion classification model to obtain deep features fs of each training sample in the first dataset and deep features ft of each training sample in the second dataset. A300 inputs fs and ft into a pre-built global domain adversarial module to calculate the global domain adversarial loss; inputs fs and ft into the softmax of the classifier, and then inputs them into a pre-built local domain adversarial module to calculate the local domain adversarial loss; A400, combined with the preset balance coefficient, balances the global domain adversarial loss and the local domain adversarial loss to obtain the global-local domain adversarial loss; A500, maps Ds and Dt to a low-dimensional space through a fully connected layer, denoted as A i and B j ; Calculate A i 、B j The similarity between them is used to calculate the probability of transferring from any element of the first matrix to any element of the second matrix. The probability of transferring from any element of the second matrix to any element of the first matrix By calculation and The product of the two gives the probability of transferring from any element of the first matrix to any element of the second matrix and then to any element of the first matrix. calculate The cross entropy between the source domain and the target domain is used as a loss to measure the similarity between the data of the same category and the target domain. Among them, A represents the feature matrix after the source domain training sample is mapped to the low-dimensional space, which is used as the first matrix, and B represents the feature matrix after the target domain training sample is mapped to the low-dimensional space, which is used as the second matrix; L visit =(V,P visit ) Among them, M ij Indicates A i 、B j The similarity between them, exp() represents the expectation, L visit Indicates the measurement loss of the similarity between the source domain and the target domain, Represents the feature sample corresponding to the matrix A, V represents P ab The transition probability conforms to the label of uniform distribution; A600, which is a weighted sum of the global-local domain adversarial loss, the loss of similarity between labeled data of the same category, and the loss of similarity between the source and target domains, to obtain the joint domain adaptation loss; A700 inputs the deep features into the classifier to obtain a classification result, which is used as a prediction result; based on the prediction result and the true classification label, calculates the classification loss using a cross-entropy loss function; sums the classification loss with the joint domain adaptation loss to obtain a total loss, and then updates the network parameters of the EEG signal emotion classification model; A800, loop steps A100-A700 until a trained EEG signal emotion classification model is obtained.

2. The method for classifying emotional features of EEG signals based on transfer learning according to claim 1, characterized in that: The global domain adversarial module and the local domain adversarial module are both discriminators constructed with multiple fully connected layers.

3. The method for classifying emotional features of EEG signals based on transfer learning according to claim 1, characterized in that: The global-local domain adversarial loss is calculated as follows: L doamin =(1-ω)L g +ωL l Among them, L doamin represents the global-local domain adversarial loss, L g represents the global domain adversarial loss, L l represents the sum of local domain adversarial losses, n s 、n t Respectively represent the number of training samples of the source domain training dataset and the target domain training dataset, x i represents the training sample, d i represents the domain label corresponding to each domain, G d Indicates gradient reversal, G f Represents the feature extractor, that is, the multi-layer perceptron, L d represents the cross entropy loss function, ω represents the balance coefficient, i represents the subscript, represents f s or f t The probability obtained after softmax processing, C represents the set of category labels, c represents a specific category label, represents the gradient reversal feature of the local domain discriminator corresponding to category c, The local domain discriminator corresponding to the c category is based on the gradient reversal feature and the label The resulting cross entropy loss.

4. The method for classifying emotional features of EEG signals based on transfer learning according to claim 3, characterized in that: The preset balance coefficient is: in, represents the local domain adversarial loss.

5. The method for classifying emotional features of EEG signals based on transfer learning according to claim 3, characterized in that: The joint domain adaptation loss is calculated as follows: L trans =L doamin +β1L walker +β2L visit L walker =H(T,P aba ) Among them, L trans represents the joint domain adaptation loss, L walker It represents the loss of similarity between data with the same label, β1 and β2 represent the preset regularization coefficients, H represents the cross entropy loss function, class() represents the category, and T represents P aba The label of the transition probability of similar elements under the condition of uniform distribution.

6. The method for classifying emotional features of EEG signals based on transfer learning according to claim 5, characterized in that: The total loss is calculated as follows: L=L trans +L y L y =H(y,A) Among them, L represents the total loss, L y Represents the classification loss, y represents the true classification label, and A represents the feature matrix after the source domain training sample is mapped to the low-dimensional space.

7. A system for classifying emotional features of EEG signals based on transfer learning, based on the method for classifying emotional features of EEG signals based on transfer learning according to any one of claims 1 to 6, characterized in that: The system includes: data acquisition module, feature extraction module, and sentiment classification module; The data acquisition module is configured to acquire emotional EEG signal data to be classified as input data; The feature extraction module is configured to extract the differential entropy features of the input data and input the differential entropy features into the multi-layer perceptron of the EEG signal emotion classification model to obtain the deep features corresponding to the input data; The emotion classification module is configured to obtain a classification result corresponding to the input data based on the deep features through a classifier of the EEG signal emotion classification model; Among them, the EEG signal emotion classification model includes a multi-layer perceptron and a classifier.

8. An electronic device, characterized in that: include: at least one processor; and a memory communicatively coupled to at least one of said processors; The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the method for classifying emotional features of EEG signals based on transfer learning as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, which are used to be executed by the computer to implement the method for classifying emotional features of EEG signals based on transfer learning as described in any one of claims 1 to 6.

Images