A method for cross-subject EEG emotion recognition based on similarity-based dynamic cue routing

Abstract

This application provides a cross-subject EEG emotion recognition method based on similarity-based dynamic cue routing, belonging to the field of EEG emotion recognition technology. The method includes: acquiring target domain samples and multiple source domain samples formed from subject EEG data and mapping them to target shared latent representation and multiple source domain shared latent representations, respectively; injecting corresponding source domain cue vectors into the source domain shared latent representation of each source domain to obtain enhanced source domain feature representations of multiple source domains; calculating the similarity between each source domain cue vector and the target shared latent representation, converting each vector similarity into sample-level source domain routing weights, and weighting and combining each source domain cue vector to generate dynamic cue; injecting the dynamic cue into the target shared latent representation to obtain enhanced target feature representations; and inputting the enhanced source domain feature representations and enhanced target feature representations of multiple source domains into a multi-branch neural network to train and construct an emotion recognition model.

Description

Technical Field

[0001] This application relates to the field of EEG emotion recognition technology, and in particular to a cross-subject EEG emotion recognition method based on similarity-based dynamic cue routing. Background Technology

[0002] To achieve emotion recognition across subjects' electroencephalograms (EEGs), traditional methods form source domains based on subjects' EEG data. Then, a neural network model is trained using either a single source domain from a single subject (single-source adaptation) or by combining multiple source domains from multiple subjects into a single source domain (multi-source adaptation). However, traditional methods struggle to simultaneously characterize the differences between multiple source domains and the correlation between source and target domains. When there are significant differences in the distribution of EEG data among different subjects, aligning the feature distributions of different source domains can easily lead to negative transfer, causing a decline in the emotion recognition model's performance in recognizing new target domains from new subjects requiring adaptation.

[0003] Traditional multi-source domain adaptation methods, while establishing independent branches for different source domains, typically only align each branch with its own target domain. That is, after each branch processes the target domain independently, predictions for new subjects are made based on the output of each branch. This lacks an explicit "source domain knowledge retrieval and invocation mechanism." In other words, existing methods usually cannot adaptively determine "which source domains should be referenced more" based on the characteristics of new target domain samples, thus making it difficult to fully utilize the source domain information most relevant to the target domain samples.

[0004] Most existing Prompt-based methods are based on domain generalization settings that are not visible during the target domain training phase, and Prompt typically operates on the raw input or structured tensor representation. Multi-source unsupervised domain adaptation scenarios, however, usually involve training with multiple source domains and an unlabeled target domain. This requires adaptively invoking relevant source domain information even when target domain samples are visible but unlabeled. Therefore, existing Prompt-based methods and multi-source unsupervised domain adaptation frameworks differ in training settings, cue application location, and target cue generation methods. Directly migrating these methods to multi-branch EEG domain adaptation scenarios can easily lead to structural coupling difficulties, high computational redundancy, and domain semantic mismatch issues.

[0005] In addition, some existing methods mainly rely on global statistical distribution alignment, such as edge distribution alignment or coarse-grained alignment, but they do not adequately consider category information, decision boundary information, and reliability differences between different branches, which can easily lead to problems such as inter-class confusion, misjudgment near the boundary, and unstable branch fusion. Summary of the Invention

[0006] This application provides a cross-subject EEG emotion recognition method based on similarity-based dynamic cue routing, which can improve the ability to simultaneously characterize the differences between multiple source domains and the correlation between source domains and target domains, and improve the recognition performance of emotion recognition models for new target domains of new subjects that need to be adapted.

[0007] This application provides a cross-subject EEG emotion recognition method based on similarity-based dynamic cue routing, the method comprising: Obtain target domain samples and source domain samples from multiple source domains based on subject EEG data, and map them into target shared latent representation and source domain shared latent representation of multiple source domains, respectively. By injecting the corresponding source domain cue vector into the source domain shared latent representation of each source domain, we obtain enhanced source domain feature representations for multiple source domains. Calculate the vector similarity between each source domain cue vector and the target shared latent representation, convert each vector similarity into a sample-level source domain routing weight, and perform a weighted combination of each source domain cue vector to generate dynamic cue; Injecting dynamic cues into the target shared latent representation yields an enhanced target feature representation; The enhanced source domain feature representations and enhanced target feature representations from multiple source domains are input into a multi-branch neural network to train and construct an emotion recognition model, and the emotion category of the target subject is output after the model is trained.

[0008] Calculate the vector similarity between each source domain cue vector and the target shared latent representation, and convert each vector similarity into sample-level source domain routing weights, including: The target shared latent representation is projected into a query vector through a linear mapping; Calculate the similarity scores between the query vector and each source domain hint vector, and normalize each similarity score into a source domain routing weight at the sample level.

[0009] Calculate the similarity scores between the query vector and each source domain suggestion vector, including: Normalize the query vector and the hint vectors of each source domain; The query vector is multiplied by each source domain suggestion vector to obtain the original similarity score. Each original similarity score is divided by the introduced learnable temperature parameter to obtain the individual similarity scores. During training, when the temperature parameter is less than 1, the difference in similarity scores between the query vector and the source domain hint vector is amplified, and when the temperature parameter is greater than 1, the difference in similarity scores between the query vector and the source domain hint vector is reduced.

[0010] Before inputting the enhanced source domain feature representations and enhanced target feature representations from multiple source domains into the multi-branch neural network, the method further includes: By injecting the same learnable joint source domain cue vector into source domain samples from multiple source domains, an enhanced joint source domain feature representation of multiple source domains is obtained; Calculate the vector similarity between each joint source domain cue vector and the target shared latent representation, convert each vector similarity into a comprehensive weight of source domain cue at each sample level, and perform a weighted combination of the joint source domain cue vectors to generate dynamic cue; Injecting dynamic cues into the target shared latent representation yields an enhanced target feature representation; The enhanced source domain feature representations, enhanced target feature representations, enhanced joint source domain feature representations, and enhanced target feature representations from multiple source domains are input into a multi-branch neural network to train and construct an emotion recognition model. After the model is trained, the emotion category of the target subject is output.

[0011] The target domain samples do not carry emotion category labels, while the source domain samples do. The target domain samples and the source domain samples from multiple source domains are respectively input into a shared feature extractor, which outputs a target shared latent representation and a source domain shared latent representation with a unified dimension. Each source domain is assigned a learnable source domain cue vector with the same dimension as the source domain shared latent representation. The injection is performed by superimposing residuals.

[0012] A multi-branch neural network comprises multiple parallel source domain branches. Enhanced source domain feature representations and enhanced target feature representations from these multiple source domains are input into the multi-branch neural network to train and construct an emotion recognition model. After training, the model outputs the emotion category of the target subject, including: The enhanced source domain feature representation and enhanced target feature representation of each source domain are input into the corresponding source domain branch. The branch feature extractor and classifier of the source domain branch extract features and make classification predictions, and output the branch sentiment category probability. Based on the branch emotion category probability, a multi-constraint loss is determined by combining multiple loss calculation methods. The multi-constraint loss is used to optimize the source domain cue vector and temperature parameter during backpropagation in the training process. The branch quality weights are calculated based on the distribution alignment loss, semantic centroid alignment loss, and conditional entropy loss in the multi-constraint loss of each source domain branch, in order to train and construct the emotion recognition model. The target domain samples are input into the emotion recognition model, and the probability of the target branch emotion category corresponding to multiple source domain branches is calculated. The probability of the target branch emotion category is then weighted and fused according to the branch quality weight of the model to obtain the total probability of each emotion category. The emotion category with the highest total probability is determined as the emotion category of the target subject.

[0013] A multi-branch neural network comprises multiple parallel source domain branches and a joint branch. Enhanced source domain feature representations, enhanced target feature representations, enhanced joint source domain feature representations, and enhanced target feature representations from multiple source domains are input into the multi-branch neural network to train and construct an emotion recognition model. After model training, the model outputs the emotion category of the target subject, including: The enhanced source domain feature representation and enhanced target feature representation of each source domain are input to the corresponding source domain branch, and the enhanced joint source domain feature representation and enhanced target feature representation of multiple source domains are input to the joint branch; Features are extracted and classification predictions are made by the branch feature extractor and classifier of the source domain branch and joint branch, and the branch sentiment category probability is output. Based on the branch emotion category probability, a multi-constraint loss is determined by combining multiple loss calculation methods. The multi-constraint loss is used to optimize the source domain cue vector, temperature parameter, and joint source domain cue vector during backpropagation in the training process. The branch quality weights are calculated based on the distribution alignment loss, semantic centroid alignment loss, and conditional entropy loss in the multi-constraint loss of each source domain branch and joint branch, in order to train and construct the emotion recognition model. The target domain samples are input into the emotion recognition model, and the probability of the target branch emotion category corresponding to multiple source domain branches and joint branches is calculated. The probability of the target branch emotion category is then weighted and fused according to the branch quality weight of the model to obtain the total probability of each emotion category. The emotion category with the highest total probability is determined as the emotion category of the target subject.

[0014] Multi-constraint losses include distribution alignment loss, semantic centroid alignment loss, conditional entropy loss, source domain classification loss, and inter-branch collaboration loss. The distribution alignment loss, semantic centroid alignment loss, and conditional entropy loss within the multi-constraint losses calculate branch quality weights, including: Normalize the three types of losses: distribution alignment loss, semantic centroid alignment loss, and conditional entropy loss.

[0015] in, and They represent the first The and the first Distribution alignment loss of each source domain branch and They represent the first The and the first Semantic centroid alignment loss for each source domain branch and The items respectively represent the first The and the first Conditional entropy loss of each source domain branch , , These are the normalized distribution alignment loss, the normalized semantic centroid alignment loss, and the normalized conditional entropy loss, respectively. To prevent constants with a denominator of zero, For the traversal index of all source domain branches; Calculate the first loss based on the normalized three types of losses. Reliability score of each source domain branch : , Then, normalize the reliability score to obtain the branch quality weights:

[0016] in, For the first Branch quality weights of each source domain branch For the number of source domains, For the first Reliability score of each source domain branch.

[0017] The branch sentiment category probability includes the category probability predicted by the source domain branch and the joint branch for the source domain sample. A source domain classification loss is introduced for each source domain branch and the joint branch. Let the i-th branch be... Each source domain branch pairs with source domain samples The output of the first The probability of class is Its real label is Then the first Classification loss for each source domain branch Defined as:

[0018] in, Indicates the first The number of source domain samples in each source domain branch. Indicates the number of emotion categories For indicator functions; For the joint branch, its input is the joint source domain consisting of all source domain samples. Let the joint branch be the input to the joint source domain samples. The output of the first The probability of class is The true emotion label is The source domain classification loss of the joint branch is... It can be represented as:

[0019] in, This represents the number of samples from the joint source domain. The total source domain classification loss is composed of the classification loss of each source domain branch and the classification loss of the joint branch. It can be represented as: , in, This represents the number of branches in the source domain and the total source domain classification loss. This is used to ensure that each branch can learn discriminative emotion category features from labeled source domain samples; The inter-branch cooperative loss is used to constrain the consistency of the predicted distributions of each source domain branch and the joint branch on the target domain samples. Let the first branch be... Each source domain branch corresponds to a sample in the target domain. The output class probability is The class probability output by the joint branch for samples in the same target domain is Then the first Inter-branch collaboration loss between source domain branches and joint branches It can be represented as:

[0020] in, This represents the KL divergence between two predicted probability distributions.

[0021] Total inter-branch coordination loss :

[0022] Among them, the total inter-branch coordination loss This is used to enable each source domain branch to learn the global discriminative information provided by the joint branch while maintaining its own source domain adaptation capability, thereby improving the consistency and stability of target domain prediction.

[0023] This application provides a cross-subject EEG emotion recognition method based on similarity-based dynamic cue routing. It maps target domain samples and multiple source domain samples into shared latent representations, eliminating individual differences in original EEG features (such as signal amplitude and noise distribution). This allows domain-specific information from different source domains to be distinguished and quantified within the same semantic space. Source domain cue vectors are constructed in this space, ensuring consistency and comparability in the encoding of domain-specific information. Then, corresponding source domain cue vectors are injected into the shared latent representation of each source domain, explicitly preserving domain-specific information at the feature level, thereby improving the ability to simultaneously characterize the differences between multiple source domains. Next, sample-level source domain routing weights are obtained by transforming the similarity between each source domain cue vector and the target shared latent representation. These sample-level source domain routing weights are then used to weight and combine the source domain cue vectors to generate dynamic cues, which are then injected into the target shared latent representation, thereby improving the correlation between the source and target domains. By improving the ability to simultaneously characterize the differences between multiple source domains and the correlation between source and target domains, negative transfer caused by information confusion or noise introduction is reduced, thereby improving the recognition performance of emotion recognition models on new target domains for new subjects that need to be adapted, so as to meet the application needs in scenarios such as auxiliary monitoring of depression or anxiety, long-term EEG emotion change trend analysis, long-term follow-up of mental health, and related intelligent health monitoring. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0025] Figure 1 A flowchart illustrating the cross-subject EEG emotion recognition method based on similarity-based dynamic prompting routing provided in this application embodiment; Figure 2 This is a system framework diagram of the cross-subject EEG emotion recognition method based on similarity-based dynamic prompting routing provided in the embodiments of this application.

[0026] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0027] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.

[0028] The technical solution of this application and how the technical solution of this application solves the technical problem are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0029] Please see Figure 1 and Figure 2 , Figure 1 This is a flowchart illustrating the cross-subject EEG emotion recognition method based on similarity-based dynamic cue routing provided in this application. Figure 2 This application provides a system framework diagram for a cross-subject EEG emotion recognition method based on similarity-based dynamic cue routing. The emotion recognition method may include the following steps: Step S120: Obtain target domain samples and source domain samples of multiple source domains formed based on the subject's EEG data, and map them into target shared latent representation and source domain shared latent representation of multiple source domains, respectively.

[0030] Specifically, the subjects refer to the target subject and multiple other subjects. A target domain is formed based on the target subject's EEG data, and this target domain includes multiple target domain samples, none of which carry emotion category labels. The EEG data of each of the other subjects forms a source domain, and each source domain includes multiple source domain samples, each carrying an emotion category label. Emotion categories include positive, neutral, negative, or happy, sad, fearful, neutral, etc.

[0031] The target domain samples are input into the shared feature extractor, which outputs the target shared latent representation. Similarly, the source domain samples from each source domain are input into the shared feature extractor, which outputs the source domain shared latent representation for each source domain, thus obtaining shared latent representations for multiple source domains. The shared feature extractor is a multilayer perceptron structure, where the source domain shared latent representation is the source domain shared latent feature vector, and the target shared latent representation is the target shared latent feature vector.

[0032] It is understandable that the shared latent representation of the target and source domains serves as a unified feature basis for subsequent steps such as cue injection, branch adaptation, and classification. At the same time, mapping the EEG features of different subjects' EEG data to the same latent space provides a unified representation basis for the subsequent construction of source domain cue vectors and the routing mechanism corresponding to the target shared latent representation.

[0033] Step S140: Inject the corresponding source domain cue vector into the source domain shared latent representation of each source domain to obtain the enhanced source domain feature representation of multiple source domains.

[0034] Specifically, a learnable source domain cue vector with the same latent representation dimension as the source domain is first set for each source domain. The source domain cue vector is used to capture the unique features of each source domain, forming a source domain cue library (e.g., ...). Figure 2 As shown, the source domain suggestion library includes source domain suggestion vectors S1Prompt for source domain 1, S2Prompt for source domain 2, etc., and then injects the corresponding source domain suggestion vector into the source domain shared latent representation for each source domain. In one implementation, the source domain suggestion vectors are generated using a random initialization method during initial setup, and each source domain corresponds to an independent learnable source domain suggestion vector. The source domain suggestion vectors have the same dimension as the source domain shared latent representation and are updated through backpropagation along with the model parameters during the training of the improved neural network model.

[0035] During training, the source domain cue vectors adaptively capture the unique features of each source domain (such as the high-frequency fluctuation pattern of subject A's EEG signal under "negative" emotions), enabling the improved neural network model to dynamically call upon the knowledge of relevant source domains through subsequent similarity calculations. This setting ensures that the source domain cue vectors of each source domain can retain domain specificity while being compatible with other modules (such as dynamic cue generation and multi-branch networks), providing personalized source domain knowledge support for cross-subject emotion recognition.

[0036] Injection refers to the use of residual superposition, which involves adding the source domain cue vector to the source domain shared latent representation obtained by the corresponding source domain sample through the shared feature extractor to obtain an enhanced source domain feature representation, rather than directly injecting the source domain cue vector into the original EEG data or the original source domain sample.

[0037] The purpose of assigning a learnable source domain cue vector, which shares the same latent representation dimension as the source domain, to each source domain is to encode the domain-specific information (domain-specific information refers to the unique EEG characteristics of each subject, such as the subject's unique emotional response patterns and EEG rhythm characteristics) in the shared latent space into learnable cue prototypes (i.e., cue vectors). This transforms "domain specificity" from implicit parameter differences into explicit and searchable cue representations. In other words, compared to the traditional multi-source domain adaptation scheme where "each source domain is independently aligned and lacks an explicit source domain invocation mechanism," this application additionally constructs a source domain cue library located in the shared latent space, allowing the specific information of each source domain to be explicitly stored in the form of cue prototypes.

[0038] The target domain samples and multiple source domain samples are mapped to a shared latent representation of a unified dimension. This space eliminates individual differences in the original EEG features (such as signal amplitude and noise distribution), enabling domain-specific information from different source domains to be distinguished and quantified in the same semantic space. Source domain cue vectors are constructed in this space, ensuring that the encoding of domain-specific information is consistent and comparable. The dimension of the source domain cue vector is consistent with the shared latent representation, and source domain features can be directly enhanced through residual superposition, so that domain-specific information is explicitly preserved at the feature level.

[0039] Step S160: Calculate the similarity between each source domain cue vector and the target shared latent representation, convert each vector similarity into a sample-level source domain routing weight, and perform a weighted combination of each source domain cue vector to generate dynamic cue.

[0040] Specifically, the calculation process for the source domain routing weights at each sample level is as follows: The target shared latent representation is projected into a query vector through linear mapping. The similarity scores between the query vector and each source domain hint vector are calculated, and each similarity score is normalized into a sample-level source domain routing weight.

[0041] The calculation process for each similarity score is as follows: The query vector and each source domain hint vector are normalized separately. Then, the query vector and each source domain hint vector are multiplied by a dot product to obtain the original similarity scores. Finally, each original similarity score is divided by the introduced learnable temperature parameter to obtain the final similarity score. During training, when the temperature parameter is less than 1, the difference in similarity scores between the query vector and the source domain hint vector is amplified; when the temperature parameter is greater than 1, the difference in similarity scores between the query vector and the source domain hint vector is reduced.

[0042] Let the target sharing potential representation be: , No. The source domain cue vectors are The number of source domains is First, the query vector is obtained through linear mapping. : , in, This represents the learnable query weight matrix, whose initial values ​​are preset. Next, the query vector and the source domain hint vectors are normalized: , in, , These represent the normalized query vector and the source domain hint vector, respectively.

[0043] Then calculate the similarity score. : ,in, This is a learnable scaling parameter. Alternatively, a temperature parameter can also be used. , is represented as: and .

[0044] The similarity scores are normalized to obtain the sample-level source domain routing weights. :

[0045] in, This represents the total score for all similarity scores.

[0046] The source domain hint vectors are weighted and combined based on the sample-level source domain routing weights to generate dynamic hints. : .

[0047] Step S180: Inject dynamic cues into the target shared latent representation to obtain an enhanced target feature representation.

[0048] Specifically, the injected features are superimposed using a residual method, meaning that the dynamic cues and the target shared latent representation are superimposed using a residual method. For example, the dynamic cues and the target shared latent representation are residually superimposed to obtain an enhanced target feature representation. :

[0049] Dynamic suggestions are generated by transforming the similarity between each source domain suggestion vector and the target shared latent representation, resulting in sample-level source domain routing weights. These weights are then used to weight and combine the source domain suggestion vectors, injecting the dynamic suggestions into the target shared latent representation. In this way, instead of assigning a fixed source domain to the target shared latent representation corresponding to a target domain sample, the system adaptively performs "soft retrieval" and "prototype interpolation" based on the relevance between the target shared latent representation and multiple source domain suggestion vectors, thus forming dynamic suggestions that more closely reflect the attributes of the target domain sample.

[0050] Compared to schemes that rely solely on branch structures or simply perform global distribution alignment, this application adds a pre-routing module before the target domain samples enter the branch adaptation process. This module consists of "target shared latent representation mapping - similarity calculation between source domain hint vectors and target shared latent representations - dynamic hints - enhanced target feature representation", thus functionally enabling sample-level source domain knowledge retrieval.

[0051] Compared to the scheme that uses multi-head cross-attention to generate target cues, this application adopts a similarity-based dynamic cue routing mechanism (i.e., step S160), which is essentially a soft retrieval and weighted interpolation of the source domain cue vector. It has a lighter structure, more direct semantics, and is more suitable for the current one-dimensional source domain shared latent representation.

[0052] Step S200: Input the enhanced source domain feature representations and enhanced target feature representations of multiple source domains into a multi-branch neural network to train and construct an emotion recognition model, and output the emotion category of the target subject after the model training is completed.

[0053] Compared to input-level source domain cue vector or structured source domain cue vector generation schemes, the source domain cue vector in this application does not act on the original EEG data input (i.e., it is not directly injected into the original EEG data or source domain samples), but acts on the shared latent representation. Therefore, it is easier to couple with the multi-branch domain adaptation framework in terms of structure, and the training process is more stable.

[0054] In the above technical solution, target domain samples and source domain samples from multiple source domains are mapped to shared latent representations, enabling domain-specific information from different source domains to be distinguished and quantified within the same semantic space. Source domain cue vectors are constructed in this space to ensure the consistency and comparability of the encoding of domain-specific information. Furthermore, a corresponding source domain cue vector is injected into the source domain shared latent representation for each source domain, explicitly preserving domain-specific information at the feature level. Next, sample-level source domain routing weights are obtained by transforming the vector similarity between each source domain cue vector and the target shared latent representation. Based on these sample-level source domain routing weights, the source domain cue vectors are weighted and combined to generate dynamic cue, which is then injected into the target shared latent representation, thereby improving the correlation modeling capability between the source and target domains. This approach reduces negative transfer caused by source domain information confusion or interference from irrelevant source domains, improving the recognition performance of emotion recognition models for new subjects' target domains, thus meeting the application needs in scenarios such as auxiliary monitoring of depression or anxiety states, long-term EEG emotion change trend analysis, long-term follow-up of mental health, and related intelligent health monitoring.

[0055] like Figure 2 As shown, in one embodiment, the multi-branch neural network includes multiple parallel source domain branches. Step S200 includes: inputting the enhanced source domain feature representations and enhanced target feature representations of the multiple source domains into the multi-branch neural network to train and construct an emotion recognition model, and outputting the emotion category of the target subject after the model training is completed, including: Step S210: Input the enhanced source domain feature representation and enhanced target feature representation of each source domain into the corresponding source domain branch. The branch feature extractor and classifier of the source domain branch extract features and make classification predictions, and output the branch sentiment category probability.

[0056] Step S220: Based on the branch emotion category probability, a multi-constraint loss is determined by combining multiple loss calculation methods. The multi-constraint loss is used to optimize the source domain cue vector and temperature parameter during backpropagation in the training process.

[0057] Specifically, the multi-constraint loss includes distribution alignment loss, semantic centroid alignment loss, conditional entropy loss, and source domain classification loss; in implementations of multi-branch neural networks that include joint branches, the multi-constraint loss also includes inter-branch collaborative loss. The loss calculation methods corresponding to each loss can be the maximum mean difference algorithm, the feature centroid distance algorithm, the conditional entropy minimization algorithm, the cross-entropy loss algorithm, and the mean squared error algorithm, respectively.

[0058] Step S230: Calculate the branch quality weights based on the distribution alignment loss, semantic centroid alignment loss, and conditional entropy loss in the multi-constraint loss of each source domain branch to construct the emotion recognition model.

[0059] Specifically, the branch reliability score is estimated based on the distribution alignment loss, semantic centroid alignment loss, and conditional entropy loss, and then the reliability score is converted into branch quality weights.

[0060] Branch quality weights are used to measure the reliability of each source domain branch's fit to the target domain samples. Therefore, they are preferably determined based on distribution alignment loss, semantic centroid alignment loss, and conditional entropy loss. Distribution alignment loss reflects the overall feature distribution difference between the source and target domains, semantic centroid alignment loss reflects the fine-grained alignment at the category level, and conditional entropy loss reflects the confidence level of the target domain prediction results. These three types of losses are directly related to the target domain fit quality and are therefore suitable as the basis for branch reliability scoring.

[0061] Normalize the three types of losses: distribution alignment loss, semantic centroid alignment loss, and conditional entropy loss.

[0062] in, and They represent the first The and the first Distribution alignment loss of each source domain branch and They represent the first The and the first Semantic centroid alignment loss for each source domain branch and The items respectively represent the first The and the first Conditional entropy loss of each source domain branch , , These are the normalized distribution alignment loss, the normalized semantic centroid alignment loss, and the normalized conditional entropy loss, respectively. To prevent constants with a denominator of zero, This provides the traversal index for all source domain branches.

[0063] Calculate the first based on the normalized loss. Reliability score of each source domain branch : , Then, normalize the reliability score to obtain the branch quality weights: , in, For the first Branch quality weights of each source domain branch For the number of source domains, For the first Reliability score of each source domain branch.

[0064] Distribution alignment loss is used to reduce the overall feature distribution difference between the source domain and the target domain; conditional entropy loss is used to improve the prediction confidence of the target domain and make the target domain samples move away from the classification decision boundary; semantic centroid alignment loss is used to strengthen category-level alignment based on pseudo-labels.

[0065] The source domain classification loss is used to maintain the ability of each source domain branch to discriminate the sentiment of labeled source domain samples. The branch sentiment category probability includes the predicted probabilities of the source domain branch and the joint branch for the source domain samples. A source domain classification loss is introduced for each source domain branch and the joint branch, let the i-th branch be the first branch. Each source domain branch pairs with source domain samples The output of the first The probability of class is The true emotion label is The number of emotion categories is Then the first The source domain classification loss for each source domain branch can be expressed as:

[0066] in, For the first The number of source domain samples in each source domain branch. This is an indicator function. When the true label of the sample belongs to the... If the value is positive, it is zero; otherwise, it is zero. Through this source domain classification loss, each source domain branch can learn the emotion category discrimination boundary based on source domain samples carrying emotion category labels.

[0067] In multi-branch neural networks, including joint branches, multiple source domain samples collectively constitute a joint source domain sample. The joint branch also calculates the source domain classification loss based on cross-entropy loss. Let the joint branch classify the joint source domain samples... The output of the first The probability of class is The true emotion label is The source domain classification loss of the joint branch can be expressed as:

[0068] in, Let be the number of samples in the joint source domain. The source domain classification loss can be composed of the classification loss of each source domain branch and the classification loss of the joint branch. The total source domain classification loss is composed of the classification loss of each source domain branch and the classification loss of the joint branch. It can be represented as: , in, This represents the number of branches in the source domain and the total source domain classification loss. This is used to ensure that each branch can learn discriminative emotion category features from labeled source domain samples; In the implementation of the joint branch, the branch sentiment category probability also includes the category probabilities predicted by the source domain branch and the joint branch for the same target domain sample. The inter-branch cooperative loss is used to constrain the consistency of the prediction distributions of each source domain branch and the joint branch on the target domain sample, so as to convey complementary information between source domain-specific information and multi-source common information. Let the... Each source domain branch corresponds to a sample in the target domain. The output class probability is The class probability output by the joint branch for samples in the same target domain is Then the first Inter-branch collaboration loss between source domain branches and joint branches It can be represented as:

[0069] in, This represents the KL divergence between two predicted probability distributions.

[0070] Total inter-branch coordination loss :

[0071] Total inter-branch coordination loss This is used to enable each source domain branch to learn the global discriminative information provided by the joint branch while maintaining its own source domain adaptation capability, thereby improving the consistency and stability of target domain prediction.

[0072] Step S240: Input the target domain sample into the emotion recognition model, calculate the probability of the target branch emotion category corresponding to multiple source domain branches, and perform weighted fusion of the target branch emotion category probability according to the branch quality weight of the model to obtain the total probability of each emotion category, and determine the emotion category with the highest total probability as the emotion category of the target subject.

[0073] Specifically, after the target domain samples are input into the model, the processing of the target domain samples is the same as that in the training phase. That is, after the target domain samples are input into the model, they are first mapped into a target shared latent representation. The dynamic cues of the model are then injected into the target shared latent representation to obtain an enhanced target feature representation. The enhanced target feature representation is input into multiple source domain branches and outputs the probability of the target branch's sentiment category. If the first The probability of the target branch sentiment category output by each source branch is: , Indicates the category of emotion. Let the target domain sample be represented. Then the total probability of each emotion category is: , Emotional categories of target participants for: .

[0074] Compared to schemes that use only a single classification output or average fusion, this embodiment adaptively allocates branch quality weights based on the branch reliability score of each branch, so that the final decision not only considers the output of multiple source domain branches, but also the current adaptation reliability of each branch to the target domain samples.

[0075] The above technical solution, with its improved neural network model, can explicitly preserve the domain-specific information of each source domain in a shared latent representation space under multi-source unsupervised domain adaptation conditions. Furthermore, it adaptively generates sample-level dynamic cues for each target domain sample based on its similarity to the cue vectors of each source domain, thereby enhancing the expression of target features. By further combining multi-branch neural network adaptation, fine-grained alignment, branch collaborative learning, and branch quality-weighted fusion, it improves the accuracy, stability, and generalization ability of emotion recognition across subject EEG data, meeting the application needs in scenarios such as auxiliary monitoring of depression or anxiety states, long-term EEG emotion change trend analysis, long-term follow-up of mental health, and related intelligent health monitoring.

[0076] In another embodiment, the multi-branch neural network further includes a joint branch. The joint branch is used to receive an enhanced joint source domain feature representation corresponding to a joint source domain sample composed of multiple source domain samples, and an enhanced target feature representation generated from target domain samples, thereby learning common information among multiple source domains.

[0077] Specifically, prior to step S200, the method further includes: Step S191: Combine the source domain samples from multiple source domains to form a joint source domain sample, and input the joint source domain sample into the shared feature extractor to obtain the joint source domain shared latent representation.

[0078] Step S192: Inject learnable joint source domain cue vectors into the joint source domain shared latent representation to obtain an enhanced joint source domain feature representation.

[0079] The joint source domain cue vector encodes common knowledge across multiple source domains, while the source domain cue vector encodes specific knowledge specific to each source domain. The joint source domain cue vector and the source domain cue vector complement each other, enabling the model to simultaneously model general emotional patterns across source domains and the unique EEG distribution characteristics of each source domain.

[0080] Step S193: Input the enhanced source domain feature representation and enhanced target feature representation of each source domain into the corresponding source domain branch, and input the enhanced joint source domain feature representation and enhanced target feature representation into the joint branch.

[0081] Step S194: Extract features and perform classification prediction through the branch feature extractor and classifier of the source domain branch and joint branch, and output the probability of multiple branch sentiment categories.

[0082] Step S195: Based on the probabilities of multiple branch sentiment categories, determine the multi-constraint loss by combining multiple loss calculation methods, and calculate the branch quality weights according to the distribution alignment loss, semantic centroid alignment loss and conditional entropy loss of each source domain branch and joint branch.

[0083] Step S196: Input the target domain sample into the trained emotion recognition model, calculate the probability of the target branch emotion category corresponding to multiple source domain branches and joint branches, and perform weighted fusion of the target branch emotion category probability according to the branch quality weight to determine the emotion category of the target subject.

[0084] In this way, the joint branch and the source domain branch run in parallel. The joint branch is used to learn common knowledge across multiple source domains, while the source domain branch is used to learn domain-specific information for each source domain. Together, they form a multi-path adaptation framework, enabling the model to simultaneously utilize cross-source domain general rules and source domain-specific patterns, avoiding the one-sided capture of information by a single branch.

[0085] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for cross-subject EEG emotion recognition based on similarity-based dynamic cue routing, characterized in that the method... include: Obtain target domain samples and source domain samples from multiple source domains based on subject EEG data, and map them into target shared latent representation and source domain shared latent representation of multiple source domains, respectively. By injecting the corresponding source domain cue vector into the source domain shared latent representation of each source domain, we obtain enhanced source domain feature representations for multiple source domains. Calculate the vector similarity between each source domain cue vector and the target shared latent representation, convert each vector similarity into a sample-level source domain routing weight, and perform a weighted combination of each source domain cue vector to generate dynamic cue; Injecting dynamic cues into the target shared latent representation yields an enhanced target feature representation; The enhanced source domain feature representations and enhanced target feature representations from multiple source domains are input into a multi-branch neural network to train and construct an emotion recognition model, and the emotion category of the target subject is output after the model is trained.

2. The method according to claim 1, characterized in that, The calculation of the vector similarity between each source domain cue vector and the target shared latent representation, and the conversion of each vector similarity into sample-level source domain routing weights, includes: The target shared latent representation is projected into a query vector through a linear mapping; Calculate the similarity scores between the query vector and each source domain hint vector, and normalize each similarity score into a source domain routing weight at the sample level.

3. The method according to claim 2, characterized in that, The calculation of similarity scores between the query vector and each source domain suggestion vector includes: Normalize the query vector and the hint vectors of each source domain; The query vector is multiplied by each source domain suggestion vector to obtain the original similarity score. Each original similarity score is divided by the introduced learnable temperature parameter to obtain the individual similarity scores. During training, when the temperature parameter is less than 1, the difference in similarity scores between the query vector and the source domain hint vector is amplified, and when the temperature parameter is greater than 1, the difference in similarity scores between the query vector and the source domain hint vector is reduced.

4. The method according to claim 1, characterized in that, Before inputting the enhanced source domain feature representations and enhanced target feature representations from multiple source domains into the multi-branch neural network, the method further includes: By injecting the same learnable joint source domain cue vector into source domain samples from multiple source domains, an enhanced joint source domain feature representation of multiple source domains is obtained; Calculate the vector similarity between each joint source domain cue vector and the target shared latent representation, convert each vector similarity into a comprehensive weight of source domain cue at each sample level, and perform a weighted combination of the joint source domain cue vectors to generate dynamic cue; Injecting dynamic cues into the target shared latent representation yields an enhanced target feature representation; The enhanced source domain feature representations, enhanced target feature representations, and enhanced joint source domain feature representations from multiple source domains are input into a multi-branch neural network consisting of multiple source domain branches and a joint branch to train and construct an emotion recognition model. After the model is trained, the model outputs the emotion category of the target subject.

5. The method according to claim 1 or 4, characterized in that, The target domain samples participate in model training during the training phase but do not carry emotion category labels, while the source domain samples carry emotion category labels. The target domain samples and source domain samples from multiple source domains are respectively input into a shared feature extractor, which outputs a target shared latent representation and multiple source domain shared latent representations with a unified dimension. Each source domain is assigned a learnable source domain cue vector with the same dimension as the source domain shared latent representation. The injection method uses residual superposition, that is, adding the source domain cue vector to the corresponding source domain shared latent representation, or adding the dynamic cue to the target shared latent representation.

6. The method according to claim 1, characterized in that, The multi-branch neural network includes multiple parallel source domain branches. The enhanced source domain feature representations and enhanced target feature representations of the multiple source domains are input into the multi-branch neural network to train and construct an emotion recognition model. After the model training is completed, the model outputs the emotion category of the target subject, including: The enhanced source domain feature representation and enhanced target feature representation of each source domain are input into the corresponding source domain branch. The branch feature extractor and classifier of the source domain branch extract features and make classification predictions, and output the branch sentiment category probability. Based on the branch emotion category probability, a multi-constraint loss is determined by combining multiple loss calculation methods. The multi-constraint loss is used to optimize the source domain cue vector and temperature parameter during backpropagation in the training process. The branch quality weights are calculated based on the distribution alignment loss, semantic centroid alignment loss, and conditional entropy loss in the multi-constraint loss of each source domain branch, in order to train and construct the emotion recognition model. The target domain samples are input into the emotion recognition model, and the probability of the target branch emotion category corresponding to multiple source domain branches is calculated. The probability of the target branch emotion category is then weighted and fused according to the branch quality weight of the model to obtain the total probability of each emotion category. The emotion category with the highest total probability is determined as the emotion category of the target subject.

7. The method according to claim 1 or 4, characterized in that, The multi-branch neural network includes multiple parallel source domain branches and a joint branch. The enhanced source domain feature representations, enhanced target feature representations, enhanced joint source domain feature representations, and enhanced target feature representations of the multiple source domains are input into the multi-branch neural network to train and construct an emotion recognition model. After the model training is completed, the model outputs the emotion category of the target subject, including: The enhanced source domain feature representation and enhanced target feature representation of each source domain are input to the corresponding source domain branch, and the enhanced joint source domain feature representation and enhanced target feature representation of multiple source domains are input to the joint branch; Features are extracted and classification predictions are made by the branch feature extractor and classifier of the source domain branch and joint branch, and the branch sentiment category probability is output. Based on the branch emotion category probability, a multi-constraint loss is determined by combining multiple loss calculation methods. The multi-constraint loss is used to optimize the source domain cue vector, temperature parameter, and joint source domain cue vector during backpropagation in the training process. The branch quality weights are calculated based on the distribution alignment loss, semantic centroid alignment loss, and conditional entropy loss in the multi-constraint loss of each source domain branch and joint branch, in order to train and construct the emotion recognition model. The target domain samples are input into the emotion recognition model, and the probability of the target branch emotion category corresponding to multiple source domain branches and joint branches is calculated. The probability of the target branch emotion category is then weighted and fused according to the branch quality weight of the model to obtain the total probability of each emotion category. The emotion category with the highest total probability is determined as the emotion category of the target subject.

8. The method according to claim 7, characterized in that, The multi-constraint loss includes distribution alignment loss, semantic centroid alignment loss, conditional entropy loss, source domain classification loss, and inter-branch collaboration loss.

9. The method according to claim 6, characterized in that, The distribution alignment loss, semantic centroid alignment loss, and conditional entropy loss in the multi-constraint loss are used to calculate branch quality weights, including: Normalize the three types of losses: distribution alignment loss, semantic centroid alignment loss, and conditional entropy loss. in, and They represent the first The and the first Distribution alignment loss of each source domain branch and They represent the first The and the first Semantic centroid alignment loss for each source domain branch and The items respectively represent the first The and the first Conditional entropy loss of each source domain branch , , These are the normalized distribution alignment loss, the normalized semantic centroid alignment loss, and the normalized conditional entropy loss, respectively. To prevent constants with a denominator of zero, For the traversal index of all source domain branches; Calculate the first loss based on the normalized three types of losses. Reliability score of each source domain branch : ， Then, normalize the reliability score to obtain the branch quality weights: ， in, For the first Branch quality weights of each source domain branch For the number of source domains, For the first Reliability score of each source domain branch.

10. The method according to claim 7, characterized in that, The multi-constraint loss also includes source domain classification loss and inter-branch collaboration loss. The branch sentiment category probability includes the category probability predicted by the source domain branch and the joint branch for the source domain sample. A source domain classification loss is introduced for each source domain branch and the joint branch. Let the first branch be... Each source domain branch pairs with source domain samples The output is the first The probability of class is Its real label is Then the first Classification loss for each source domain branch Defined as: in, Indicates the first The number of source domain samples in each source domain branch. Indicates the number of emotion categories. For indicator functions; For the joint branch, its input includes the joint source domain consisting of all source domain samples. Let the joint branch be the input to the joint source domain samples. The output of the first The probability of class is The true emotion label is The source domain classification loss of the joint branch is... It can be represented as: in, This represents the number of samples from the joint source domain. The total source domain classification loss is composed of the classification loss of each source domain branch and the classification loss of the joint branch. It can be represented as: ， in, This represents the number of branches in the source domain and the total source domain classification loss. This is used to ensure that each branch can learn discriminative emotion category features from labeled source domain samples; The inter-branch collaborative loss is used to constrain the consistency of the predicted distributions of each source domain branch and the joint branch on the target domain samples. Let the first branch be... Each source domain branch corresponds to a sample in the target domain. The output class probability is The class probability output by the joint branch for samples in the same target domain is Then the first Inter-branch collaboration loss between source domain branches and joint branches It can be represented as: in, This represents the KL divergence between two predicted probability distributions; Total inter-branch coordination loss : Among them, the total inter-branch coordination loss This is used to enable each source domain branch to learn the global discriminative information provided by the joint branch while maintaining its own source domain adaptation capability, thereby improving the consistency and stability of target domain prediction.

Images