Cross-database electroencephalogram emotion recognition optimization method based on prototype representation pairwise learning

Abstract

The invention discloses a cross-database electroencephalogram emotion recognition optimization method based on prototype representation pairwise learning, comprising: constructing a prototype representation pairwise learning model, the prototype representation pairwise learning model comprising a feature discriminator module, a prototype feature extraction module and a classifier module; respectively extracting sample features of the source domain sample and the target domain sample by adopting a feature identifier module; using a prototype feature extraction module to extract prototype features; a classifier module is adopted to obtain interaction features according to the sample features and the prototype features, similarity of sample pairs is calculated according to the interaction features through a pairwise learning method, labels of the sample pairs are obtained, iteration updating is conducted on the prototype representation pairwise learning model according to the similarity of the sample pairs and the labels of the sample pairs, and the prototype representation pairwise learning model is obtained. Calculating emotion classification performance according to the interaction features; and optimizing the prototype to characterize the pairwise learning model. The robustness and generalization can be improved, the emotion recognition is optimized, and the accuracy of the emotion recognition is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electroencephalogram (EEG) analysis, and in particular to a cross-database EEG emotion recognition optimization method based on prototype representation paired learning. Background Art

[0002] Traditional emotion recognition mainly relies on non-physiological signals such as facial expressions, voice signals, and body movements, which are highly subjective and have average accuracy. As a physiological signal, EEG signals have the characteristics of being difficult to disguise, and have advantages such as superior real-time and objectivity. They can provide more direct and objective clues for understanding and evaluating emotional states. Therefore, they are increasingly attracting the attention of researchers in different fields such as computer science, neuroscience, and signal processing. In recent years, researchers have been committed to developing emotion recognition models based on electroencephalograms (EEGs). Using transfer learning and domain adaptation techniques, they have shown very good performance in cross-individual emotion recognition tasks within a single database, and have also been successfully applied to large public EEG emotion datasets such as SEED and SEED-IV. However, due to the large differences between different fields (databases), traditional EEG emotion recognition mainly focuses on intra-individual or cross-task. When dealing with cross-database EEG emotion recognition tasks, in addition to dealing with individual differences, they also have to face many factors such as environmental differences and equipment differences in EEG acquisition in different databases, which leads to a significant decline in the performance of these emotion recognition methods. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a cross-database EEG emotion recognition optimization method based on prototype representation paired learning to improve the accuracy of emotion recognition.

[0004] To solve the above technical problems, the purpose of the present invention is achieved through the following technical solutions: to provide a cross-database EEG emotion recognition optimization method based on prototype representation paired learning, comprising the following steps: constructing a prototype representation paired learning model, the prototype representation paired learning model includes a feature discriminator module, a prototype feature extraction module and a classifier module; using the feature discriminator module to extract sample features of source domain samples and target domain samples respectively; using the prototype feature extraction module to extract prototype features; using the classifier module to obtain interaction features based on sample features and prototype features, calculating the similarity of sample pairs based on the interaction features through a paired learning method, obtaining the label of the sample pair, iteratively updating the prototype representation paired learning model based on the similarity of the sample pairs and the label of the sample pairs, and calculating the emotion classification performance based on the interaction features; optimizing the prototype representation paired learning model.

[0005] The beneficial technical effect of the present invention is that the cross-database EEG emotion recognition optimization method based on prototype representation paired learning of the present invention constructs a prototype representation paired learning model including a feature discriminator module, a prototype feature extraction module and a classifier module, and uses the feature discriminator module to extract sample features of source domain samples and target domain samples respectively; uses the prototype feature extraction module to extract prototype features; uses the classifier module to obtain interaction features based on sample features and prototype features, calculates the similarity of sample pairs based on the interaction features through a paired learning method, obtains the label of the sample pair, iteratively updates the prototype representation paired learning model based on the similarity of the sample pairs and the label of the sample pairs, and calculates the emotion classification performance based on the interaction features; optimizes the prototype representation paired learning model to improve robustness and generalization, and optimizes emotion recognition by obtaining the similarity between each sample in the feature space and designing a specific model structure and training method, thereby improving the accuracy of emotion recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0007] Figure 1 A flowchart of a cross-database EEG emotion recognition optimization method based on prototype representation paired learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0008] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0009] See also Figure 1 As shown, Figure 1 A schematic flow chart of a cross-database EEG emotion recognition optimization method based on prototype representation paired learning provided by an embodiment of the present invention, wherein the cross-database EEG emotion recognition optimization method based on prototype representation paired learning comprises the following steps:

[0010] Step S10, constructing a prototype representation paired learning model, wherein the prototype representation paired learning model includes a feature identifier module, a prototype feature extraction module and a classifier module;

[0011] Step S20: using a feature identifier module to extract sample features of the source domain sample and the target domain sample respectively;

[0012] Step S30, extracting prototype features using a prototype feature extraction module;

[0013] Step S40, using a classifier module to obtain interactive features based on sample features and prototype features, calculating the similarity of sample pairs based on the interactive features through a paired learning method, obtaining labels of the sample pairs, iteratively updating the prototype representation paired learning model based on the similarity of the sample pairs and the labels of the sample pairs, and calculating the emotion classification performance based on the interactive features; wherein the labels of the sample pairs include true labels and pseudo labels, which are the results of determining whether the two samples of the sample pair belong to the same emotion category, and have a value of only 0 or 1. When the value of the label of the sample pair is 1, it indicates that the two samples of the sample pair belong to the same emotion category. When the value of the label of the sample pair is 0, it indicates that the two samples of the sample pair do not belong to the same emotion category.

[0014] Step S50: Optimize the prototype representation paired learning model.

[0015] Among them, the samples are EEG data in different EEG emotion databases. The cross-database EEG emotion recognition optimization method based on prototype representation paired learning constructs a prototype representation paired learning model including a feature discriminator module, a prototype feature extraction module and a classifier module, and uses the feature discriminator module to extract sample features of source domain samples and target domain samples respectively; uses the prototype feature extraction module to extract prototype features; uses the classifier module to obtain interaction features based on sample features and prototype features, calculates the similarity of sample pairs based on the interaction features through a paired learning method, obtains the label of the sample pair, iteratively updates the prototype representation paired learning model based on the similarity of the sample pairs and the label of the sample pairs, and calculates the emotion classification performance based on the interaction features; optimizes the prototype representation paired learning model to improve robustness and generalization, and optimizes emotion recognition by obtaining the similarity between each sample in the feature space and designing a specific model structure and training method, thereby improving the accuracy of emotion recognition.

[0016] Preferably, the step S20 is specifically:

[0017] The feature extractor of the feature discriminator module is used to extract corresponding sample features for the source domain samples and the target domain samples respectively, that is, the feature extractor of the feature discriminator module is used to extract the domain-invariant sample features of the source domain samples and the target domain samples, and the feature discriminator of the feature discriminator module is used to distinguish the domain to which the sample features belong, that is, the feature discriminator of the feature discriminator module is used to distinguish whether the sample corresponding to the sample features belongs to the source domain or the target domain.

[0018] Among them, the feature discriminator module may include a feature extractor and a feature discriminator. The feature extractor is used to extract sample features according to the sample. The feature extractor includes two hidden layers. The feature discriminator is used to distinguish whether the sample corresponding to the sample feature belongs to the source domain or the target domain. The feature extractor can characterize the sample features based on the domain adaptive neural network (Domain Adaptation Neural Network, DANN) using a domain adversarial training method. By minimizing the difference between the sample features of the source domain samples and the target domain samples through domain adaptation and adversarial training, and extracting the relevant features of the domain invariant attributes, the influence of individual differences in EEG signals can be effectively alleviated, and the robustness and generalization ability of the model can be improved. The difference between the source domain samples and the target domain samples can be minimized by calculating the domain adversarial loss function. The domain adversarial loss function can be a binary cross entropy loss function, which is expressed by formula (1):

[0019]

[0020] In the formula, l disc (·) represents the domain adversarial loss function, X S represents the source domain sample set, X T represents the target domain sample set, represents the i-th sample in the source domain sample set, represents the jth sample in the target domain sample set, n represents the number of samples in the source domain, m represents the number of samples in the target domain, D(·) represents the domain discriminator, and f(·) represents the feature extraction function. represents the sample features of the i-th sample in the source domain sample set, Represents the sample features of the jth sample in the target domain sample set.

[0021] Preferably, the step S30 is specifically as follows:

[0022] The prototype feature extraction module is used to extract prototype features according to the sample features of the source domain samples.

[0023] Among them, each emotion category from the source domain or the target domain corresponds to a prototype feature. Samples belonging to the same emotion category have the same prototype feature. The prototype feature is the most essential attribute of all sample features of the emotion category. The sample features belonging to the emotion category are distributed around the prototype feature. The prototype feature is the centroid of all sample features of the emotion category, that is, the average value of all sample features of the corresponding domain in the emotion category. Extracting the prototype feature requires the guidance of the emotion category. Since the target domain emotion category is invisible, the prototype feature is extracted from all samples in the source domain. By extracting the prototype feature, the robustness and generalization can be improved. Then the specific steps of step S30 are:

[0024] The prototype feature extraction module is used to obtain the average vector of sample features of source domain samples corresponding to each emotion category as the prototype feature corresponding to each emotion category.

[0025] Specifically, the average vector of the prototype features corresponding to each emotion category is calculated using formula (2):

[0026]

[0027] In the formula, ψ 1:z Represents the set of prototype features of each emotion category, ψ 1:z ={ψ1,ψ2,ψ3,...,ψ z}, the element ψ in the set · is the prototype feature of each emotion category, z represents the number of emotion categories, represents the i-th sample with emotion category c in the source domain, represents the number of samples with emotion category c, f(·) represents the feature extraction function, then Represents the sample features of the i-th sample with emotion category c in the source domain.

[0028] Preferably, the step S40 of using a classifier module to obtain interactive features according to sample features and prototype features is specifically:

[0029] The bilinear transformation strategy is used to characterize the interaction between the sample features and the prototype features to obtain the interactive features, wherein the interactive features are characterized by the sample features and the prototype features. The interactive features can be expressed by formula (3):

[0030] Γ=(F·ψ c ·θ)(3)

[0031] In the formula, Γ represents the interaction feature, · represents the inner product operation of the matrix, F represents the sample feature, which is obtained by calculating the sample through the feature extraction function f(·), ψ c represents the prototype feature of emotion category c, θ represents the bilinear transformation matrix, where the parameters of the bilinear transformation matrix are continuously iteratively updated during the model training process. Among them, the interaction feature Γ represents the probability that the sample belongs to each emotion category. For example, when the interaction feature of the sample is [0.22, 0.65, 0.13], since the probability of the second position is the largest, the sample will be classified into the second emotion category. The emotion recognition accuracy can be calculated based on the interaction feature.

[0032] Specifically, in the step S40 of calculating the similarity of the sample pair according to the interaction feature by the paired learning method, the similarity of the sample pair is obtained by calculating the cosine similarity according to the interaction feature, and the target loss function of the paired learning method can be expressed by formula (4):

[0033]

[0034] In the formula, l pair represents the objective loss function of the paired learning method, a represents the number of samples, Γ represents the interaction feature, i and j represent the sequence numbers of the two samples in the sample pair, and φ ij (·) represents the similarity between two samples in a sample pair, that is, the similarity between two samples x in a sample pair i and x j The probability of belonging to the same emotion category is between 0 and 1. The closer it is to 1, the more likely the two samples in the sample pair are to belong to the same emotion category. The closer it is to 0, the more likely the two samples in the sample pair are not to belong to the same emotion category. ij The label of the sample pair is the result of judging whether the two samples of the sample pair belong to the same emotion category. Its value is only 0 or 1. ij =1 indicates that the two samples of the sample pair belong to the same emotion category, μ ij =0 indicates that the two samples in the sample pair do not belong to the same emotion category.

[0035] Among them, the source domain belongs to supervised learning, and the emotion label is visible. In the source domain, the label of the sample pair is the real label. The label of the sample pair in the source domain can be calculated using formula (5):

[0036]

[0037] In the formula, R(·) represents the rounding function, y s represents the true label of the source domain sample after one-bit efficient encoding, φ ij (·) represents the similarity between two samples in a sample pair, Γ s Represents the interaction features of source domain samples.

[0038] The target domain belongs to unsupervised learning, and the emotion label is invisible. In the target domain, the label of the sample pair is a pseudo label. An adaptive nonlinear dynamic update threshold is used to calculate the label of the sample pair in the target domain, which is expressed by formula (6):

[0039]

[0040] In the formula, φ ij (·) represents the similarity between two samples in a sample pair, Γ t represents the interaction features of target domain samples, η u and η dThey represent the upper and lower limits of the thresholds updated by adaptive nonlinear dynamics. When the similarity of the sample pair exceeds the upper limit, it can be considered that the sample pair belongs to the same emotion category, and the label of the sample pair is 1. When the similarity of the sample pair is lower than the lower limit, it can be considered that the sample pair does not belong to the same emotion category, and the label of the sample pair is 0, and 0<η d ≤η u <1.

[0041] Among them, in order to improve the robustness and generalization performance of the model and improve the quality of model training data, in the target domain, sample pairs between the upper and lower thresholds do not participate in model training, but with the continuous iteration of the model, the threshold will be adaptively nonlinearly dynamically updated, the upper threshold will gradually decrease, and the lower threshold will gradually increase, so that more and more sample pairs that meet the requirements participate in the training.

[0042] The updating process of the adaptive nonlinear dynamic updating threshold is expressed by formula (7):

[0043]

[0044] In the formula, Epoch_size represents the number of training rounds. represents the upper threshold value at time t, represents the upper threshold value at time t-1, represents the lower limit of the threshold at time t, Indicates the lower limit of the threshold at time t-1. The threshold at the current moment is affected by the threshold at the previous moment.

[0045] The total objective loss function of the prototype representation paired learning model is calculated using formula (8):

[0046]

[0047] Where, L pl represents the total objective loss function of the prototype representation pairwise learning model, represents the pairwise learning loss value of the source domain, represents the pairwise learning loss value of the target domain, l disc represents the domain adversarial loss value of the feature discriminator module, α and β represent hyperparameters. and the pairwise learning loss value of the target domain The domain adversarial loss value l of the feature discriminator module can be calculated according to formula (4) to formula (7) respectively. discIt can be calculated according to formula (1). The prototype representation paired learning model is trained through the target loss function of the prototype representation paired learning model. In the training process, domain adversarial and paired learning methods are used to improve the emotion recognition performance of the model. In addition, in the training process of the model, the labels of the sample pairs in the target domain are completely invisible, so the wrong labels and noise labels in the target domain have limited impact on the model performance.

[0048] Preferably, the step S50, i.e. optimizing the prototype representation paired learning model, is specifically:

[0049] According to the sample features, the local maximum mean difference (LMMD) method is used to measure the similarity between the source domain samples and the target domain samples in the feature space, and the feature discriminator module is optimized to obtain a prototype representation paired learning model with the local maximum mean difference. Among them, the local maximum mean difference unbiased estimation objective function is expressed by formula (9):

[0050]

[0051] In the formula, represents the feature mapping function, X S and X T They represent the source domain sample set and the target domain sample set respectively. Hk represents the reproducible Hilbert space with feature kernel k, which is the feature mapping space. Feature kernel k is a distance function that defines the distance between the sample pair in the Hilbert space, the source domain sample and the target domain sample. z represents the number of emotion categories, and n c represents the number of all samples with emotion category c in the source domain, m c represents the number of all samples with emotion category c in the target domain, represents the probability weight of the i-th source domain sample belonging to the source domain when the given emotion category is c, represents the i-th sample in the source domain sample set, represents the probability weight that the jth target domain sample belongs to the target domain when the given emotion category is c, represents the jth sample in the target domain sample set, and LMMD represents the local maximum mean difference. By obtaining the probability that each sample belongs to the corresponding domain in each emotion category, the feature contribution of the two domains can be balanced, and the features of the target domain can be learned more effectively. The sum of the probability weights of all samples belonging to a certain emotion category in the source domain is always 1, that is, The sum of the probability weights of all samples belonging to a certain emotion category in the target domain is always 1, that is, Weighing the source domain samples and the target domain samples, that is, evaluating the similarity of the samples based on the distance between the spatial distributions of the sample features, calculates the distance between the samples in the feature space, and the closer they are, the more similar they are. Measuring the similarity metric of the source domain samples and the target domain samples in the feature space by the local maximum average difference method can reduce the distance between the subdomains of the same emotion category between the source domain and the target domain, and achieve alignment of the subdomains of the source domain and the subdomains of the target domain in the feature space. The classifier of the classifier module of the prototype representation paired learning model with the local maximum average difference is the Ada classifier, which is a classifier using the gradient descent algorithm with the Adam adaptive learning rate.

[0052] Preferably, the step S50, i.e. optimizing the prototype representation paired learning model, is specifically:

[0053] The contrast domain difference (CDD) is calculated according to the sample features, and the feature discriminator module is optimized to obtain a prototype representation paired learning model with contrast domain difference. The objective function of contrast domain difference is expressed by formula (10):

[0054]

[0055] Where z represents the number of emotion categories, Hk represents the reproducible Hilbert space with feature kernel k,

[0056] represents the feature mapping function, d cc (·) represents the intra-domain difference between samples of the same emotion category, d cc ′(·) represents the inter-domain difference between samples of different emotion categories. The inter-domain difference and the intra-domain difference are optimized in the opposite direction, so that the sample features of the same emotion category are close to each other in the feature space, and the sample features of different categories are far away from each other. represents the set of predicted labels for samples in the target domain, CDD represents contrast domain difference. The classifier of the classifier module of the prototype representation pair learning model with contrast domain difference is an Ada classifier, and the Ada classifier is a classifier using the gradient descent algorithm with Adam adaptive learning rate.

[0057] Specifically, the classifier module of the prototype characterization paired learning model includes an Ada classifier and an Rms classifier, wherein the Ada classifier is a classifier that adopts a gradient descent algorithm with an Adam adaptive learning rate, and the Rms classifier is a classifier that adopts an RMSprop gradient descent algorithm.

[0058] Preferably, the step S50, i.e. optimizing the prototype representation paired learning model, comprises:

[0059] Optimize the classifier module.

[0060] Specifically, the optimized classifier module includes:

[0061] Perform supervised learning on the feature discriminator module and the classifier module, train the prototype representation paired learning model, so that the model reserves basic classification parameters, enables the model to obtain task-specific basic emotion recognition performance, and calculates the basic training target loss function of the prototype representation paired learning model;

[0062] Among them, the basic training objective loss function of the prototype representation paired learning model is calculated using formula (11):

[0063]

[0064] Where L1 represents the basic training objective loss function of the prototype representation pairwise learning model, l disc represents the domain adversarial loss of the feature discriminator module, represents the pairwise learning loss value of the source domain, and They represent the pairwise learning loss values ​​of the target domain of the Ada classifier and the target domain of the Rms classifier, respectively. α, β, and γ represent hyperparameters used to assign weights to each loss value. Among them, the pairwise learning loss value of the source domain Pairwise learning loss value for the target domain of the Ada classifier and the pairwise learning loss value of the target domain of the Rms classifier The domain adversarial loss value l of the feature discriminator module can be calculated according to formula (4) to formula (7) respectively. disc It can be calculated according to formula (1);

[0065] Lock the parameters of the feature discriminator module, train the classifier module, and calculate the classifier difference according to formula (12):

[0066]

[0067] In the formula, l diff (X T ) represents the difference loss between classifiers, Mean represents the average value, X T represents the target domain sample set, x t represents the target domain sample, |·| represents the absolute value function, φ(·) represents the similarity between two samples in the sample pair, Γ Ada and Γ Rms They represent the prediction probabilities of samples by Ada classifier and Rms classifier respectively, and the interaction features are used as the prediction probability of samples.

[0068] According to formula (13), the classifier difference is maximized:

[0069]

[0070] In the formula, min means taking the minimum value, L1 represents the basic training target loss function of the prototype representation pair learning model, and C Ada and C Rms They represent Ada classifier and Rms classifier respectively, max represents the maximum value, l diff (X T ) represents the classifier difference, l diff (X T ) represents the difference loss between classifiers, X T represents the target domain sample set, L2 represents the target loss function of the classifier difference, and L2 is always greater than 0. By training the classifier module to maximize the classifier difference, more controversial samples distributed on the classification boundary are obtained to improve the accuracy of emotion recognition.

[0071] Lock the parameters of the classifier module and train the feature discriminator module to minimize the sample feature distribution of the source domain samples and the target domain samples according to formula (14):

[0072]

[0073] Where L3 represents the target loss function, m represents the number of samples in the target domain, z represents the number of emotion categories, and |·| represents the absolute value function. represents the i-th target domain sample, represents the predicted probability that the sample is of emotion category c in the Ada classifier, represents the predicted probability of the sample being of emotion category c in the Rms classifier, F represents the sample feature, Represents minimized feature distribution. The feature distribution of samples in the source and target domains is minimized by adjusting the mapping of the target domain samples in the feature discriminator module to minimize the difference between the two classifiers.

[0074] In summary, the cross-database EEG emotion recognition optimization method based on prototype representation paired learning of the present invention constructs a prototype representation paired learning model including a feature discriminator module, a prototype feature extraction module and a classifier module, and uses the feature discriminator module to extract sample features of source domain samples and target domain samples respectively; uses the prototype feature extraction module to extract prototype features; uses the classifier module to obtain interaction features based on sample features and prototype features, calculates the similarity of sample pairs based on the interaction features through a paired learning method, obtains the labels of the sample pairs, iteratively updates the prototype representation paired learning model based on the similarity of the sample pairs and the labels of the sample pairs, and calculates the emotion classification performance based on the interaction features; optimizes the prototype representation paired learning model to improve robustness and generalization, and optimizes emotion recognition by obtaining the similarity between each sample in the feature space and designing a specific model structure and training method, thereby improving the accuracy of emotion recognition.

[0075] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A cross-database EEG emotion recognition optimization method based on prototype representation paired learning, characterized in that: The following steps are involved: Constructing a prototype representation paired learning model, wherein the prototype representation paired learning model includes a feature discriminator module, a prototype feature extraction module and a classifier module; The feature discriminator module is used to extract sample features of source domain samples and target domain samples respectively; The prototype feature extraction module is used to extract the prototype features; The classifier module is used to obtain the interaction features based on the sample features and the prototype features. The similarity of the sample pairs is calculated based on the interaction features through the paired learning method to obtain the labels of the sample pairs. The prototype representation paired learning model is iteratively updated based on the similarity of the sample pairs and the labels of the sample pairs, and the emotion classification performance is calculated based on the interaction features. Optimizing prototype representations for paired learning models.

2. The cross-database EEG emotion recognition optimization method based on prototype representation paired learning according to claim 1 is characterized in that: The feature identifier module is used to extract sample features of source domain samples and target domain samples respectively, specifically: The feature extractor of the feature discriminator module is used to extract corresponding sample features from the source domain samples and the target domain samples respectively, and the domain to which the sample features belong is distinguished by the feature discriminator of the feature discriminator module.

3. The cross-database EEG emotion recognition optimization method based on prototype representation paired learning according to claim 2 is characterized in that: The prototype feature extraction module is used to extract the prototype features, specifically: The prototype feature extraction module is used to extract prototype features according to the sample features of the source domain samples.

4. The cross-database EEG emotion recognition optimization method based on prototype representation paired learning according to claim 1 is characterized in that: The classifier module is used to obtain the interactive features according to the sample features and the prototype features, specifically: The bilinear transformation strategy is used to characterize the interaction between sample features and prototype features to obtain interactive features. The interactive features are expressed by the following formula: C=(F·ψ c ·i) In the formula, Γ represents the interaction feature, · represents the inner product operation of the matrix, F represents the sample feature, ψ c represents the prototype feature of emotion category c, and θ represents the bilinear transformation matrix.

5. The cross-database EEG emotion recognition optimization method based on prototype representation paired learning according to claim 4 is characterized in that: In the method of calculating the similarity of sample pairs according to the interaction features by using the paired learning method, the target loss function of the paired learning method is expressed by the following formula: In the formula, l pair represents the objective loss function of the paired learning method, a represents the number of samples, Γ represents the interaction feature, i and j represent the sequence numbers of the two samples in the sample pair, and φ ij (·) represents the similarity between two samples in a sample pair, μ ij Represents the label of the sample pair.

6. The cross-database EEG emotion recognition optimization method based on prototype representation paired learning according to claim 1 is characterized in that: The optimized prototype represents a paired learning model, specifically: According to the sample features, the local maximum average difference method is used to measure the similarity metric of the source domain samples and the target domain samples in the feature space, and the feature discriminator module is optimized to obtain a prototype representation paired learning model with the local maximum average difference; among which, the local maximum average difference unbiased estimation objective function is expressed by the following formula: In the formula, represents the feature mapping function, X S and X T They represent the source domain sample set and the target domain sample set respectively, Hk represents the reproducible Hilbert space with feature kernel k, z represents the number of emotion categories, n c represents the number of all samples with emotion category c in the source domain, m c represents the number of all samples with emotion category c in the target domain, represents the probability weight of the i-th source domain sample belonging to the source domain when the given emotion category is c, represents the i-th sample in the source domain sample set, represents the probability weight that the jth target domain sample belongs to the target domain when the given emotion category is c, Represents the jth sample in the target domain sample set.

7. The cross-database EEG emotion recognition optimization method based on prototype representation paired learning according to claim 1 is characterized in that: The optimized prototype represents a paired learning model, specifically: The contrast domain difference is calculated according to the sample features, and the feature discriminator module is optimized to obtain a prototype representation paired learning model with contrast domain difference. The objective function of the contrast domain difference is expressed by the following formula: Where z represents the number of emotion categories, Hk represents the reproducible Hilbert space with feature kernel k, represents the feature mapping function, d cc (·) represents the intra-domain difference between samples of the same emotion category, d cc ′(·) represents the inter-domain difference between samples of different emotion categories, Represents the set of predicted labels for target domain samples.

8. The cross-database EEG emotion recognition optimization method based on prototype representation paired learning according to claim 1 is characterized in that: The classifier modules of the prototype characterization paired learning model include Ada classifier and Rms classifier.

9. The cross-database EEG emotion recognition optimization method based on prototype representation paired learning according to claim 8, characterized in that: The optimization prototype represents a paired learning model, including: Optimize the classifier module.

10. The cross-database EEG emotion recognition optimization method based on prototype representation paired learning according to claim 9, characterized in that: The optimized classifier module comprises: The feature discriminator module and the classifier module are supervisedly learned, the prototype representation paired learning model is trained, and the basic training target loss function of the prototype representation paired learning model is calculated; wherein, the basic training target loss function of the prototype representation paired learning model is calculated using the following formula: Where L1 represents the basic training objective loss function of the prototype representation paired learning model, l disc represents the domain adversarial loss of the feature discriminator module, represents the pairwise learning loss value of the source domain, and denote the pairwise learning loss values ​​of the target domain of the Ada classifier and the pairwise learning loss values ​​of the target domain of the Rms classifier, respectively. α, β, and γ denote hyperparameters used to assign weights to each loss value. Lock the parameters of the feature discriminator module, train the classifier module, and calculate the classifier difference according to the following formula: In the formula, l diff (X T ) represents the difference loss between classifiers, Mean represents the average value, X T represents the target domain sample set, x t represents the target domain sample, |·| represents the absolute value function, φ(·) represents the similarity between two samples in the sample pair, Γ Ada and Γ Rms Respectively represent the prediction probabilities of the samples by the Ada classifier and the Rms classifier; Maximize the classifier variance according to the following formula: In the formula, min means taking the minimum value, L1 represents the basic training objective loss function of the prototype representation paired learning model, and C Ada and C Rms They represent Ada classifier and Rms classifier respectively, max represents the maximum value, l diff (X T ) represents the classifier difference, l diff (X T ) represents the difference loss between classifiers, X T represents the target domain sample set, L2 represents the target loss function of the classifier difference; Lock the parameters of the classifier module and train the feature discriminator module to minimize the sample feature distribution of source domain samples and target domain samples according to the following formula: Where L3 represents the target loss function, m represents the number of samples in the target domain, z represents the number of emotion categories, and |·| represents the absolute value function. represents the i-th target domain sample, represents the predicted probability that the sample is of emotion category c in the Ada classifier, represents the predicted probability of the sample being of emotion category c in the Rms classifier, F represents the sample feature, represents the minimization of feature distribution.

Images