Cross-domain emotion recognition method of attention mechanism based on electroencephalogram

Abstract

The invention discloses a cross-domain emotion recognition method based on an attention mechanism of electroencephalogram, which comprises the following steps: extracting domain invariant features of the electroencephalogram, and extracting features of a specific domain by using a multi-layer sensor; aligning edge distribution and condition distribution from different fields by combining the maximum mean value difference loss and the correlation similarity loss; electroencephalogram emotion classification is carried out by using a domain specific feature classifier, and difference loss is adopted, so that the accuracy of a high cross-domain electroencephalogram emotion recognition result is obtained. According to the method, the attention mechanism based on the convolution block is introduced to automatically identify and focus on the most critical feature of the cross-domain task, and the MMD loss and the association similarity loss are combined, so that the cross-domain emotion recognition accuracy is greatly improved. According to the method, the accuracy is improved by 10%-30% in a cross-subject scene, the accuracy is improved by 10%-20% in a cross-time-period scene, and particularly, the method has the advantages in the aspects of improving the target domain recognition accuracy and reducing the distribution difference between the source domain and the target domain.

Description

Technical Field

[0001] The present invention belongs to the technical field of emotion recognition, and in particular relates to a cross-domain emotion recognition method based on an electroencephalogram (EEG) attention mechanism. Background Art

[0002] Emotion recognition enables computers to understand and respond to human emotional states, and is the basis for achieving more natural, efficient and empathetic human-computer interaction. Emotion recognition has a wide range of applications in the diagnosis and treatment of medical diseases, traffic safety, market research, and other fields. As a tool that directly reflects brain activity, EEG provides a new research perspective for emotion recognition. When users try to conceal their true feelings, traditional behavior-based emotion recognition methods may be limited, while biometric methods such as EEG can provide deeper emotion recognition. In-depth research on the application of EEG in emotion recognition can better understand the neural mechanism of emotion generation and improve the accuracy and efficiency of emotion recognition. Although research on EEG emotion recognition has made remarkable achievements, it still faces many challenges. As a highly complex biological signal, EEG signals have characteristics such as high dimensionality and strong nonlinearity. Signals in different frequency bands have different meanings and are easily affected by individual differences, electrode layout and external interference. In addition, obtaining high-quality EEG signals requires precise experimental settings and professional equipment. Providing sufficient labeled data for deep learning algorithms is both time-consuming and resource-intensive. Therefore, it is urgent to develop efficient neural network models to achieve accurate recognition of EEG signals.

[0003] Real-life situations often involve emotion recognition scenarios across subjects and time periods. Even when performing the same task on different subjects, the features of the EEG data generated are very different, which results in poor generalization of the high-performance model trained on existing subjects when tested on new subjects. Different models have been proposed for domain adaptation methods, such as a domain adaptation neural network based on a deep adversarial network, which trains the feature extractor in the direction of deceiving domain identification by maximizing the domain discrimination loss; a new domain adaptation method for emotion recognition, which aims to generalize the model across individuals and time periods, minimizes classification errors by optimizing the neural network model, and makes the source data and target data similar in potential representation. Through the application of these domain adaptation related methods, researchers are gradually solving the problem of individual differences in EEG data, thereby improving the generalization ability of the model on different subjects. However, current research is insufficient in capturing effective features between the target domain and the source domain, and is also unsatisfactory in efforts to narrow the inter-domain differences to improve the average classification accuracy. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a cross-domain emotion recognition method based on EEG attention mechanism. A deep over-parameterized convolution structure is used as a feature extractor, and a lightweight network is used to capture rich EEG features. The introduced attention mechanism can automatically identify and focus on those features that are most critical for cross-domain tasks. By jointly adopting MMD loss and associated similarity loss, AMSJDA not only aligns the source domain and the target domain in terms of feature degree, but also makes the prediction more consistent, thereby improving the classification effect of the model on the target domain. Finally, the difference loss is used to calculate the prediction results of multiple target domains, making the classification results more accurate. Compared with the existing multi-source domain adaptation model, this model can effectively narrow the differences between domains, improve the consistency of target predictions, and achieve high-accuracy recognition tasks in cross-subject and cross-time scenarios.

[0005] Technical solution: A cross-domain emotion recognition method based on EEG attention mechanism of the present invention comprises the following steps:

[0006] Step 1: Receive N independent source domain EEG datasets and one target domain EEG dataset and extract common features through a shared common feature extraction module with CBAM attention mechanism;

[0007] Step 2: Send the extracted common features to their respective domain-specific feature extractors;

[0008] Step 3: Minimize the maximum mean difference distance and associated similarity between common features to narrow the differences between different domains and align the source domain features with the target domain features;

[0009] Step 4: Introduce the target domain features and all source domain features into their respective classifiers to generate corresponding classification predictions;

[0010] Step 5: Use the results of the source domain to calculate the classification loss, feed the target domain data into each source domain classifier, generate multiple target domain prediction results and calculate the difference loss;

[0011] Step 6: Take the average value of the target domain prediction results as the final output of the model to achieve emotion recognition in cross-time scenarios.

[0012] Furthermore, step 1 is specifically as follows: receiving N independent source domain data sets and a target domain dataset {X T}, using the deep over-parameterized convolutional module as a common feature extractor, introducing the CBAM attention mechanism, by receiving the intermediate features F∈R of the deep over-parameterized convolutional layer C×H×W As input, we first calculate the channel attention and assign different weights to each dimension;

[0013]

[0014] Among them, F'∈R C ' ×H×W represents the output after channel attention, C' represents the channel information after the channel attention weight has been assigned to C, represents element-wise multiplication;

[0015] M c (F) is the calculation result of the channel attention weight of F, which can be expressed as:

[0016] M c (F)=σ(W1(W0(AvgPool(F)))+W1(W0(MaxPool(F)))) (2)

[0017] Among them, σ represents the activation function Sigmoid, AvgPool represents the average pooling operation, MaxPool represents the maximum pooling operation, W0∈R C / r×C , W1∈R C×C / r , r is a constant;

[0018] Then, a map of spatial attention is generated based on the internal spatial relations characterized by the following features;

[0019]

[0020] where F'∈R C '×H' ×W ', where H' and W' represent the representation after the spatial attention weights have been assigned to H and W;

[0021] M s (F') is the calculation of the spatial attention weight of F, which can be expressed as:

[0022] M s (F') = σ(f n×n [AvgPool(F');MaxPool(F')]) (4)

[0023] Where n represents the size of the convolution kernel, and [AvgPool(F'); MaxPool(F')] represents the cascade of average pooling and maximum pooling results.

[0024] Furthermore, step 2 is specifically as follows: using a multi-layer perceptron as a domain-specific feature extractor, the data of each source domain and target domain are passed through their corresponding domain-specific feature extractors, the data are mapped into a unique latent space, and fine, domain-specific features are extracted from each branch.

[0025] Furthermore, step 3 is as follows: first, MMD is used to estimate the distance between the two domains. The formula of MMD is expressed as:

[0026]

[0027] Among them, N S and N T denote the number of samples in the source domain and the target domain respectively, φ(·) is the feature mapping function that maps the original data to the high-dimensional feature space, and the MMD loss identifies and reduces the statistical differences between the two domains by comparing the average feature distributions of the two domains. The association similarity is used to improve the consistency of label prediction between the source domain and the target domain. The association similarity loss is expressed as:

[0028] L asso =γ1·L walker +γ2·L visit (6)

[0029] Where γ1 and γ2 are weight factors. walker and L visit ;

[0030] Suppose a set of source domain data is A and the target domain data is B. First, we need to calculate the i To each target domain sample B j The transition probability P s→t .P s→t The calculation formula is expressed as:

[0031]

[0032] Among them, (similarity(A i ,B j )) represents the source domain sample A i and target domain sample B j The similarity measure between them is usually the dot product, and then the sample B from the target domain is calculated. j Back to source domain sample A i The transition probability P t→s (j,i);

[0033]

[0034] Using the above two transition probabilities, we can calculate the i To the target domain sample B j Back to the source domain sample A' i The two-step cycle transition probability, that is, A i To B j The transition probability and B j Back to A' i The transition probability is multiplied by , and then all target domain samples B j sum;

[0035] P s→t→s (i,i')=∑ j P s→t (i,j)P t→s j,i') (9)

[0036] L walker It can be expressed as:

[0037] L walker =KL(log(P s→t→s +∈)||T s ) (10)

[0038] Here, ∈ is a small constant added for numerical stability and is chosen to be 1e -8 , KL(·) is the KL divergence, T s is defined as:

[0039]

[0040] According to P s→t (i, j) calculates the average probability P of the target sample being visited visit (j);

[0041]

[0042] Among them, N s is the total number of source domain samples;

[0043] Define an ideal access probability distribution V, in which the probability of each target sample being accessed is equal, and V is expressed as:

[0044]

[0045] Among them, N t is the total number of target domain samples, Vj) represents the probability of any target domain sample j being visited in an ideal situation, that is, each sample is considered equally;

[0046] By calculating P visit The difference between (j) and Vj) is used to define L visit The loss can be expressed as:

[0047]

[0048] where KL(·) is the KL divergence.

[0049] Furthermore, step 4 is specifically as follows: using the features extracted from the domain-specific feature extractor to predict the results, in the domain-specific classifier, each source domain corresponds to N single Softmax classifiers.

[0050] Furthermore, step 5 is as follows: for each trained classifier, a cross entropy loss function is selected to estimate the classification loss, and a difference loss metric is introduced to make the predictions of N classifiers converge. The classification loss can be expressed as:

[0051]

[0052] in, represents the predicted label, represents the true label, J(·) represents the cross entropy loss function, and the difference loss can be expressed as:

[0053]

[0054] in, represents the model's predicted output for the i-th sample in the target domain, and represents the true label or target of the jth sample in the target domain, and the total loss function can be expressed as:

[0055] L=L cls +αL MMD +βL disc +γL assoc (17)

[0056] During the training process, the α and γ coefficients are dynamically adjusted, and the expressions are as follows:

[0057]

[0058] Where i represents the current iteration number, k and c are preset parameters, which determine the rate and position of γ changing with the iteration number i.

[0059] The present invention also discloses a computer device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method of the present invention.

[0060] The present invention also discloses a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the steps of the method of the present invention are implemented.

[0061] The present invention also discloses a computer program product, comprising a computer program / instruction, which implements the steps of the method of the present invention when executed by a processor.

[0062] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0063] 1. The present invention proposes to use a deep over-parameterized convolutional module as this common feature extractor, and further introduces the CBAM attention mechanism. The introduction of the attention mechanism enables the model to automatically identify the most critical features in each domain, thereby improving the model's adaptability and generalization ability to domain data. Compared with the method of simply merging all existing data into a single source domain, the method proposed in the present invention adopts a multi-source domain adaptation architecture. By combining the deep over-parameterized convolutional network with the attention mechanism, it can more effectively process multi-source domain data and ensure that the model can achieve good performance between various domains.

[0064] 2. The present invention combines MMD loss and associated similarity loss at the same time. Compared with other multi-source domain adaptation models that only consider the marginal distribution of features in reducing inter-domain differences, the present invention can reduce the distribution difference between the source domain and the target domain and promote label prediction consistency. The associated similarity strategy not only prompts the model to learn category-identifying and transferable feature representations between the source domain and the target domain, but also ensures that the model has a comprehensive understanding of the overall structure of the target domain. This method is particularly suitable for domain adaptation tasks where the source domain and target domain data distribution may be significantly different. It helps the model to maintain high classification accuracy for source domain samples while also being able to better generalize to new, unseen target domain data.

[0065] 3. The strategy of dynamically adjusting the loss coefficient adopted by the present invention can help the model converge better and prevent instability caused by some loss terms being too strong in the early stage of training. Through such a setting, the model can focus on learning basic features in the early stage and gradually shift its attention to more complex or task-specific features in the later stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 Flow chart of the CBAM attention mechanism of the system of the present invention;

[0067] Figure 2 is a model flow chart of the system of the present invention;

[0068] Figure 3 This is the T-SNE visualization without the AMSJDA model of the system of the present invention;

[0069] Figure 4 This is a T-SNE visualization of the AMSJDA model using the system of the present invention. DETAILED DESCRIPTION

[0070] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0071] (I) Implementation process of emotion recognition model

[0072] Step 1: Receive N independent source domain EEG datasets and an EEG dataset of the target domain {X T}, using the deep over-parameterized convolution module as a common feature extractor, and on this basis, further introducing the CBAM attention mechanism. Figure 1 As shown, by receiving the intermediate features F∈R of the deep over-parameterized convolutional layer C×H×W As input, the channel attention is first calculated, assigning different weights to each dimension.

[0073]

[0074] Among them, F'∈R C'×H×W represents the output after channel attention, C' represents the channel information after the channel attention weight has been assigned to C, Represents element-wise multiplication.

[0075] M c (F) is the calculation result of the channel attention weight of F, which can be expressed as:

[0076] M c (F)=σ(W1(W0(AvgPool(F)))+W1(W0(MaxPool(F)))) (2)

[0077] Where σ represents the activation function Sigmoid, AvgPool represents the average pooling operation, and MaxPool represents the maximum pooling operation. C / r×C , W1∈R C×C / r , r is a constant, and the purpose of dimensionality transformation is to reduce parameter overhead.

[0078] Then, a spatial attention map is generated according to the internal spatial relations characterized by the following features.

[0079]

[0080] where F'∈R C '×H'×W', where H' and W' represent the representation after the spatial attention weights have been assigned to H and W.

[0081] M s (F') is the calculation of the spatial attention weight of F, which can be expressed as:

[0082] M s (F') = σ(f n×n [AvgPool(F');MaxPool(F')]) (4)

[0083] Where n represents the size of the convolution kernel, and [AvgPool(F'); MaxPool(F')] represents the cascade of average pooling and maximum pooling results.

[0084] Step 2: Using multi-layer perceptron as domain-specific feature extractor, the data of each source domain and target domain are passed through its corresponding domain-specific feature extractor to map the data into a unique latent space and extract fine-grained, domain-specific features from each branch.

[0085] Step 3: First, use MMD to estimate the distance between the two domains. The formula of MMD is:

[0086]

[0087] Among them, N S and N T denotes the number of samples in the source domain and the target domain, respectively. φ(·) is the feature mapping function that maps the original data to a high-dimensional feature space. The MMD loss identifies and reduces the statistical differences between the two domains by comparing their average feature distributions. This helps ensure that the model as a whole can capture the common feature attributes of the source and target domains.

[0088] We further use the association similarity to improve the consistency of label prediction between the source domain and the target domain. The association similarity loss is expressed as:

[0089] L asso =γ1·L walker +γ2·L visit (6)

[0090] Where γ1 and γ2 are weight factors. walker Ensure that the prediction of the source domain sample is close in probability to the prediction of the sample after passing through the target domain and then returning to the source domain. visit Ensures that every sample in the target domain is considered with equal probability, thus preventing the model from ignoring target domain samples that are difficult to classify. asso Combining the advantages of both, the model can learn to help classification while also being able to better generalize to new, unseen target domain data. walker and L visit .

[0091] Suppose a set of source domain data is A and the target domain data is B. First, we need to calculate the i To each target domain sample B j The transition probability P s→t .P s→t The calculation formula is expressed as:

[0092]

[0093] Among them, (similarity(A i ,B j )) represents the source domain sample A i and target domain sample B j The similarity measure between the target domain samples is usually the dot product. The denominator ensures that the similarity measure of all target domain samples is normalized so that the sum of the transition probabilities is 1. Similarly, the calculation of the similarity measure from the target domain sample B j Back to source domain sample A i The transition probability P t→s (j,i).

[0094]

[0095] Next, the above two transition probabilities are used to calculate the i To the target domain sample B j Back to the source domain sample A' i The two-step cycle transition probability. i To B j The transition probability and B j Back to A' i The transition probability is multiplied by , and then all possible target domain samples B j Sum.

[0096] P s→t→s (i,i')=∑ j P s→t (i,j)P t→s j,i') (9)

[0097] L walker Aims to make P s→t→s The category consistency distribution T between the source domain samples in the ideal case s Matching can be expressed as:

[0098] L walker =KL(log(P s→t→s +∈)||T s ) (10)

[0099] Here, ∈ is a small constant added for numerical stability and is chosen to be 1e -8 , KL(·) is the KL divergence. T s is defined as:

[0100]

[0101] According to P s→t (i, j) calculates the average probability P of the target sample being visited visit (j).

[0102]

[0103] Among them, N s is the total number of source domain samples.

[0104] In order to make the model evenly consider each sample of the target domain, an ideal access probability distribution V is defined, in which the probability of each target sample being accessed is equal. V is expressed as:

[0105]

[0106] Among them, N t is the total number of target domain samples. V(j) represents the probability of any target domain sample j being visited in an ideal situation, that is, each sample is considered equally.

[0107] By calculating P visit The difference between (j) and Vj) is used to define L visit The loss can be expressed as:

[0108]

[0109] where KL(·) is the KL divergence.

[0110] Step 4: Use the features extracted from the domain-specific feature extractor to predict the outcome. In the domain-specific classifier, each source domain corresponds to N single Softmax classifiers.

[0111] Step 5: For each trained classifier, choose the cross entropy loss function to estimate the classification loss. A metric called difference loss is introduced to make the predictions of N classifiers converge. The classification loss can be expressed as:

[0112]

[0113] in, represents the predicted label, Represents the true label, and J(·) represents the cross entropy loss function. The difference loss is used to further improve the classification accuracy. The difference loss can be expressed as:

[0114]

[0115] in, represents the model's predicted output for the i-th sample in the target domain, and represents the true label or target of the jth sample in the target domain. The overall loss function can be expressed as:

[0116] L=L cls +αL MMD+βL disc +γL assoc (17)

[0117] Among them, L MMD We can obtain domain-invariant features for each pair of source and target domains, L assoc Further improve the feature similarity between known subjects and target subjects and minimize L cls This can enable the classifier trained on the source domain to achieve higher classification accuracy, minimizing L disc This can make the classification results more convergent. During the training process, the α and γ coefficients are dynamically adjusted, and the expressions are as follows:

[0118]

[0119] Where i represents the current iteration number, k and c are preset parameters, which determine the rate and position of γ changing with the iteration number i.

[0120] (II) Experimental verification

[0121] The cross-domain emotion recognition task can be divided into two main scenarios: cross-subject transfer and cross-period transfer. In the cross-subject transfer scenario, the EEG data of a new subject in a certain period is defined as the target domain, while the EEG data of other subjects constitute the source domain. For cross-period transfer, the data of a subject in the previous period is used as the source domain, and the data collected by the subject in the subsequent period is regarded as the target domain.

[0122] The collected dataset covers 3 periods and 15 subjects. For the cross-period scenario, the data of one subject in two periods are selected as the source domain for cross-period transfer, and the data in the remaining period is used as the target domain. For the cross-subject scenario, the data of 14 subjects in the same period are used as the source domain for cross-subject transfer, and the data of the remaining subject is used as the target domain.

[0123] The batch size is set to 64, the number of iterations is set to 200, and the coefficient of MMD loss is set to α. The coefficient of difference loss β is set to α / 100. The c in the coefficient formula of association similarity loss is set to 5, and the k value adopted by the model is 0.05. walker and L visit The coefficients γ1 and γ2 are set to 1 and 0.6 respectively.

[0124] The performance of the model on the SEED and SEED-IV datasets is listed in Table 1. This shows that the model proposed in the present invention can achieve good accuracy in the cross-domain emotion recognition scenario.

[0125] Table 1 Performance of AMSJDA model

[0126]

[0127] In order to better illustrate the CBAM attention mechanism of the present invention, the attention module for the source domain data is removed separately, and the results are shown in Table 2:

[0128] Table 2 Effect of CBAM attention mechanism in the experiment

[0129]

[0130] As can be seen from Table 2, for cross-subject experiments, the performance on the SEED dataset and SEED-IV dataset has been significantly improved, with an improvement of 1.52% and 2.87% respectively. For the emotion recognition experiment across time periods, the addition of CBAM also produced a certain performance improvement, 1.38% and 0.99% respectively. From these results, it can be seen that for the SEED dataset, the introduction of the CBAM attention mechanism significantly improved the performance. For the SEED-IV dataset, the use of CBAM also improved the performance, but the improvement was not as significant as the SEED setting.

[0131] In order to illustrate the MMD module of the present invention, different metric experiments were selected, and the results are shown in Table 3.

[0132] Table 3 Effects of different statistical distribution difference measures in multi-source domain adaptation experiments

[0133]

[0134] From the results, we can see that for cross-subject experiments, in the SEED dataset experimental setting, the MMD distance achieved the highest performance indicator of 91.18%. In the SEED-IV experimental setting, the same metric distance was tested, where the MMD distance and CORAL distance performed similarly, while the Wasserstein distance lagged slightly behind. For cross-period experiments, the effect of narrowing the gap between domains using the three metric distances is similar to that of cross-subject experiments, but under the baseline condition without any metric distance, although it has decreased, it is not as obvious as in cross-subject experiments.

[0135] In order to illustrate the association similarity module of the present invention, this module is also removed separately, and the results are shown in Table 4.

[0136] Table 4 Effect of association similarity in multi-source domain adaptation experiment

[0137]

[0138] As can be seen from the table, on the SEED dataset, after removing the associated similarity, the cross-subject recognition accuracy slightly dropped from 91.18% to 90.65%, and the cross-period recognition accuracy also slightly dropped from 87.43% to 86.08%. On the SEED-IV dataset, the cross-subject recognition accuracy dropped from 62.98% to 61.34%, and the cross-period recognition accuracy dropped from 66.89% to 66.55%.

[0139] In the cross-subject scenario, 100 EEG samples were randomly selected from each subject (i.e., different domains) for T-SNE visualization analysis. The cross-subject scenario of the SEED dataset was selected for plotting, with different colors used to mark each source domain, while the target domain was represented in black. To improve the clarity of the graph, the target samples were made transparent to reduce overlap. Figure 3 , Figure 4 By comparison, we can see that in the AMSJDA method, the target domain data points marked in black are close to and overlap with the data points in each source domain in the feature space. By carefully adjusting the distribution between the source and target domains, the data points in the target domain are no longer isolated, but are relatively integrated with the data points in each source domain in space and are more widely and evenly distributed.

Claims

1. A cross-domain emotion recognition method based on EEG attention mechanism, characterized in that: The steps include: Step 1: Receive N independent source domain EEG datasets and one target domain EEG dataset and extract common features through a shared common feature extraction module with CBAM attention mechanism; Step 2: Send the extracted common features to their respective domain-specific feature extractors; Step 3: Minimize the maximum mean difference distance and associated similarity between common features to narrow the differences between different domains and align the source domain features with the target domain features; Step 4: Introduce the target domain features and all source domain features into their respective classifiers to generate corresponding classification predictions; Step 5: Use the results of the source domain to calculate the classification loss, feed the target domain data into each source domain classifier, generate multiple target domain prediction results and calculate the difference loss; Step 6: Take the average value of the target domain prediction results as the final output of the model to achieve emotion recognition in cross-time scenarios.

2. According to claim 1, a cross-domain emotion recognition method based on EEG attention mechanism is characterized in that: Step 1 is as follows: Receive N independent source domain datasets and a target domain dataset {X T }, using the deep over-parameterized convolutional module as a common feature extractor, introducing the CBAM attention mechanism, by receiving the intermediate features F∈R of the deep over-parameterized convolutional layer C×H×W As input, we first calculate the channel attention and assign different weights to each dimension; Among them, F'∈R C ' ×H×W represents the output after channel attention, C' represents the channel information after the channel attention weight has been assigned to C, Represents element-wise multiplication; M c (F) is the calculation result of the channel attention weight of F, which can be expressed as: M c (F)=σ(W1(W0(AvgPool(F)))+W1(W0(MaxPool(F)))) (2) Among them, σ represents the activation function Sigmoid, AvgPool represents the average pooling operation, MaxPool represents the maximum pooling operation, W0∈R C / r×C , W1∈R C×C / r , r is a constant; Then, a map of spatial attention is generated based on the internal spatial relations characterized by the following features; where F'∈R C '×H'×W', where H' and W' represent the representation after the spatial attention weights have been assigned to H and W; M s (F') is the calculation of the spatial attention weight of F, which can be expressed as: M s (F')=σ(f n×n [AvgPool(F');MaxPool(F')]) (4) Where n represents the size of the convolution kernel, and [AvgPool(F'); MaxPool(F')] represents the cascade of average pooling and maximum pooling results.

3. According to claim 1, a cross-domain emotion recognition method based on EEG attention mechanism is characterized in that: Step 2 is as follows: Using a multi-layer perceptron as a domain-specific feature extractor, the data of each source domain and target domain are passed through their corresponding domain-specific feature extractors to map the data into a unique latent space and extract fine, domain-specific features from each branch.

4. According to claim 1, a cross-domain emotion recognition method based on EEG attention mechanism is characterized in that: Step 3 is as follows: First, use MMD to estimate the distance between the two domains. The formula of MMD is expressed as: Among them, N S and N T denote the number of samples in the source domain and the target domain respectively, φ(·) is the feature mapping function that maps the original data to the high-dimensional feature space, and the MMD loss identifies and reduces the statistical differences between the two domains by comparing the average feature distributions of the two domains. The association similarity is used to improve the consistency of label prediction between the source domain and the target domain. The association similarity loss is expressed as: L asso =γ1·L walker +γ2·L visit (6) Among them, γ1, γ2 are weight factors; Suppose a set of source domain data is A and the target domain data is B. First, we need to calculate the i To each target domain sample B j The transition probability P s→t , P s→t The calculation formula is expressed as: Among them, (similarity(A i ,B j )) represents the source domain sample A i and target domain sample B j The similarity measure between them is usually the dot product, and then the sample B from the target domain is calculated. j Back to source domain sample A i The transition probability P t→s (j,i); Using the above two transition probabilities, we can calculate the i To the target domain sample B j Back to the source domain sample A' i The two-step cycle transition probability, that is, A i To B j The transition probability and B j Back to A' i The transition probability is multiplied by , and then all target domain samples B j sum; P s→t→s (i,i')=∑ j P s→t (i,j)P t→s j,i') (9) L walker It can be expressed as: L walker =KL(log(P s→t→s +∈)||T s ) (10) Here, ∈ is a small constant added for numerical stability and is chosen to be 1e -8 , KL(·) is the KL divergence, T s is defined as: According to P s→t (i, j) calculates the average probability P of the target sample being visited visit (j); Among them, N s is the total number of source domain samples; Define an ideal access probability distribution V, in which the probability of each target sample being accessed is equal, and V is expressed as: Among them, N t is the total number of target domain samples, Vj) represents the probability of any target domain sample j being visited in an ideal situation, that is, each sample is considered equally; By calculating P visit The difference between (j) and Vj) is used to define L visit The loss can be expressed as: where KL(·) is the KL divergence.

5. According to claim 1, a cross-domain emotion recognition method based on EEG attention mechanism is characterized in that: Step 4 is specifically as follows: Use the features extracted from the domain-specific feature extractor to predict the results. In the domain-specific classifier, each source domain corresponds to N single Softmax classifiers.

6. The cross-domain emotion recognition method based on EEG attention mechanism according to claim 1 is characterized in that: Step 5 is as follows: for each trained classifier, select the cross entropy loss function to estimate the classification loss, introduce the difference loss metric, and make the predictions of N classifiers converge. The classification loss can be expressed as: in, represents the predicted label, represents the true label, J(·) represents the cross entropy loss function, and the difference loss can be expressed as: in, represents the model's predicted output for the i-th sample in the target domain, and represents the true label or target of the jth sample in the target domain, and the total loss function can be expressed as: L=L cls +αL MMD +βL disc +γL assoc (17) During the training process, the α and γ coefficients are dynamically adjusted, and the expressions are as follows: Where i represents the current iteration number, k and c are preset parameters, which determine the rate and position of γ changing with the iteration number i.

7. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method of claim 1.

8. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method according to claim 1 are implemented.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to claim 1 are implemented.

Images