A multi-source manifold embedding feature selection method based on electroencephalogram mutual information

Abstract

The application discloses a multi-source manifold embedding feature selection method based on electroencephalogram mutual information, mainly comprising the following steps: (1) extracting differential entropy features of electroencephalogram according to frequency bands; (2) screening multiple source domain data and selecting high-quality source domains; (3) transforming electroencephalogram signal features to a manifold space and further extracting manifold features; (4) performing correlation and redundancy analysis on the electroencephalogram manifold features and reducing feature dimensions; (5) migrating the manifold features after dimension reduction, learning a final classifier for predicting labels; and (6) according to the classifier result, weighting and fusing multiple sets of identification results of a target domain to obtain a final classification result. Compared with a traditional method, the method improves source domain quality and feature quality, reduces calculation complexity and improves generalization performance and accuracy of a classifier model.

Description

Technical Field

[0001] This invention belongs to the research field of motor control mechanism of the nervous system, and mainly involves EEG signal preprocessing, EEG signal manifold feature extraction, manifold feature selection, and multi-source domain transfer framework design. Specifically, it refers to a multi-source manifold embedding feature selection method based on EEG mutual information. Background Technology

[0002] Emotions are a uniquely human physiological activity, encompassing expression, recognition, and transformation. They reflect the psychological and physiological responses to external stimuli and can be measured and identified using scientific methods. Utilizing computers for human emotion recognition is a crucial component of artificial intelligence, cognitive science, and affective brain-computer interfaces (aBCI). People expect machines to communicate with them based on human emotions. Therefore, emotion recognition plays a vital role in achieving this goal and has significant practical value in various applications, such as human health emotional care and patient monitoring supported by artificial intelligence technology.

[0003] In aBCI (Aspect-Based Interaction), commonly used input signals for emotion recognition include video, text, audio, and physiological signals. Compared to these signals, electroencephalograms (EEGs) offer better reliability and accuracy, more realistically reflecting an individual's emotional state, making them the most widely used input signal in aBCI. A typical aBCI paradigm operates as follows: First, an emotional stimulus evoking a specific emotion is presented to the user, and EEG signals are recorded according to the desired emotion. Then, EEG data features are extracted from the recorded signals, and a classifier is trained using the selected features and emotion labels. In the subsequent aBCI, the trained classifier is used to perform real-time emotion classification based on EEG signals. Many researchers have reported satisfactory classification performance under this paradigm. However, the use of aBCI is still limited by several factors. Specifically, EEG signals exhibit significant non-stationarity and individual variability, with large differences in the distribution of EEG data among different subjects. Even within the same subject, the distribution of EEG data often differs at different time points. Traditional machine learning methods require prior assumptions that the distributions of training and test data are independent and identically distributed. However, EEG signals do not always satisfy this assumption, resulting in poor performance for these methods in emotion recognition. Furthermore, EEG electrodes are susceptible to noise interference. To reduce training time for subjects and improve accuracy and adaptability, and to design and implement EEG signal analysis models with strong adaptability and high accuracy in emotion recognition, many research teams have begun to study transfer learning theories and methods, seeking a universal algorithm model applicable to all subjects to solve the common key fundamental scientific problems urgently needing to be addressed in the practical application of aBCI systems.

[0004] Transfer learning is a machine learning technique that aims to extract common knowledge from one or more source tasks and apply that knowledge to a related target task. Specifically, transfer learning in emotion recognition uses a source domain (EEG data from other users) to help a target domain (EEG data from a new user) learn. One of its main tasks is to reduce the data distribution differences between the source and target domains through mapping.

[0005] In recent years, transfer learning has been widely applied in the aBCI field. Several researchers have proposed various domain adaptation methods. PANSJ et al. proposed transfer components analysis (TCA), which solves the mapping problem in the form of a kernel function, projecting source and target domain data into a new subspace to reduce distributional differences. Joint distribution adaptation (JDA) considers the marginal and conditional distributions between different domains, representing an improvement on TCA. Wang et al. pointed out that in practical applications, marginal and conditional distributions are often treated equally, while their relative importance is not utilized; therefore, they proposed balanced distribution adaptation to adaptively utilize the importance of differences between marginal and conditional distributions. Manifold embedded distribution alignment (MEDA) performs dynamic distribution alignment on the Grassmann manifold and then learns a domain-invariant classifier to avoid feature distortion. Traditional transfer learning methods aim to solve the single-source-to-single-source-domain transfer problem. In experiments, we can see that even using the simplest transfer learning algorithm, a good source domain contributes to achieving very high classification accuracy; therefore, the quality of the source domain is crucial. However, in reality, we are likely to have multiple source domains. Just as BCI devices often have a lot of previously used labeled data, multi-source transfer learning in emotion recognition usually achieves better recognition accuracy than single-source transfer learning. When multiple source domains exist, good source domains are more likely to be included. However, in multi-source transfer learning, many aBCI-related works tend to merge all source domains into one domain, which means that EEG data from all subjects need to be used. In some practical applications, EEG data from some subjects with poor relevance to the target data may not be suitable for transfer, meaning these source domains are not good source domains. Forcing their use will result in negative transfer. Models trained in this way will not have good generalization performance. Therefore, in multi-source transfer learning, it is necessary to select appropriate knowledge output sources. Thus, a source domain selection method is needed to determine whether a source domain is suitable for transfer to the target domain, including rank of domains (ROD) (Gong et al.), domain transferability estimation (DTE) (Zhang & Wu, 2020), and domain similarity rank (Sun et al.). Summary of the Invention

[0006] This invention addresses the problems of existing EEG transfer learning methods, such as negative transfer and high redundancy of extracted EEG features. It proposes a multi-source manifold embedding feature selection method based on EEG mutual information, which can reduce the need for new labeled data, improve the quality of the source domain, reduce the sample distribution difference between the source and target domains, and perform multi-source EEG transfer learning.

[0007] To achieve the above objectives, the method of the present invention mainly includes the following steps:

[0008] Step (1) Preprocess the EEG signal and extract the EEG differential entropy features according to frequency band.

[0009] Step (2): Filter the data from multiple source domains and select high-quality source domains;

[0010] Step (3) transforms the EEG signal features into a manifold space and further extracts the manifold features.

[0011] Step (4) involves performing correlation and redundancy analysis on the electroencephalogram (EEG) features to reduce the feature dimensionality.

[0012] Step (5) involves transferring the dimensionality-reduced manifold features to learn the final classifier for predicting labels.

[0013] Step (6): Based on the classifier results, the multi-group recognition results of the target domain are weighted and fused to obtain the final classification result.

[0014] As a preferred method, when selecting the source domain, a classifier is pre-trained using data from each source domain and then applied to a small amount of target domain data with existing labels for selection. The specific steps are as follows:

[0015] Step 2-1: For the differential entropy features of the EEG signal in each target domain, divide it into labeled target domain data. and unlabeled target domain data

[0016] Step 2-2: For each source domain A Sp Let p = 1, 2, 3, ..., n, and train each group to obtain a classifier, a support vector machine (SVM). Then, ... Perform classification tests and rank the SVMs based on their classification accuracy.

[0017] Steps 2-3: Based on the ranking results, select the data trained by the classifiers with the highest accuracy as suitable source domains for subsequent transfer learning.

[0018] Preferably, step 4 includes:

[0019] Step 4-1: Based on the EEG waveform feature x of the source domain data and its corresponding label category c, calculate the correlation D between the feature and the label, and the correlation R between the features:

[0020]

[0021]

[0022] Where I(·,·) is the mutual information size, and S is the feature set.

[0023] Step 4-2: Unlike using formula (3) as the evaluation function to guide feature subset selection for dimensionality reduction, a new evaluation function formula (4) is defined by introducing an indicator vector β and parameter k.

[0024] maxφ(D,R)=DR (3)

[0025]

[0026] Where k is the final feature dimension, β = [β1β2...β] n ], where n is the number of features in the original feature set, β i The closer β is to 1, the more important the feature in that dimension is. i A value of 0 indicates that the i-th feature was not selected.

[0027] Step 4-3: For the objective function (4) in the previous step, use the incremental search method to maximize a quadratic function, solve for the indicator vector β and the final feature dimension k, and select a new source domain feature dataset F′ from the source domain and target domain data according to the above parameters. S and the target domain feature dataset F′ T .

[0028] Preferably, step 6 includes:

[0029] After obtaining the recognition accuracy of multiple sets of target domain data for multiple sets of source domain data, the recognition accuracy of each classifier is used as a weight to perform weighted fusion on the final predicted label. The weighted fusion calculation method for each sample is as shown in formula (5):

[0030]

[0031] Where n is the index of the source domain, and w is the weight. This indicates that the sample's preliminary result from n source domains is the sum of the weights of class c. The category label corresponding to the largest sum of weights is the final identification label result.

[0032] Compared with existing transfer learning methods, this invention has the following advantages:

[0033] Traditional EEG transfer learning methods do not fully utilize data information from multiple source domains, and single-source domain transfer learning is less effective than multi-source domain transfer learning. Furthermore, existing multi-source domain transfer learning methods often directly integrate multiple source domains into a single large source domain, failing to consider the existence of low-quality source domains, leading to insufficient generalization ability of the learning model and negative transfer. Simultaneously, EEG features have high redundancy and high feature dimensionality, resulting in high computational costs. To address these issues, this invention proposes a novel multi-source manifold embedding feature selection method based on EEG mutual information. This method is simple and intuitive, faster and more effective than ordinary unsupervised selection methods, and further reduces the dimensionality of EEG manifold features. While improving the quality of source domains, it also further improves the quality of EEG features, reduces feature redundancy, and lowers computational complexity, making it an effective multi-source transfer learning framework. Attached Figure Description

[0034] Figure 1 This is a flowchart of a method according to an embodiment of the present invention.

[0035] Figure 2 This is a diagram illustrating the algorithm framework of the method of this invention.

[0036] Figure 3 The effect of the number of source domains on the accuracy curve of emotion recognition for test subjects.

[0037] Figure 4 This paper compares the recognition accuracy of the method of this invention with that of various domain adaptive methods on the SEED dataset.

[0038] Figure 5 This paper compares the recognition accuracy of the method of this invention with that of various domain adaptive methods on the DEAP dataset. Detailed Implementation

[0039] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a diagram of the algorithm framework. The following description is for illustrative and explanatory purposes only and does not constitute any limitation on the invention.

[0040] Example 1

[0041] The datasets used in this paper are the SEED dataset and the DEAP dataset. Specific descriptions are as follows:

[0042] (1) SEED is an open-source emotional EEG dataset provided by Professor Lü Baoliang's team at Shanghai Jiao Tong University. The SEED dataset records EEG data from 15 subjects (7 men and 8 women, with an average age of 25 years) while watching different types of movie video clips. Each subject participated in 3 experiments, with each experiment spaced approximately one week apart. Each experiment included 15 emotion induction experiments, inducing positive, neutral, and negative emotions respectively. Each experiment included a 5-second cue, a short stimulus, a 45-second self-assessment, and a 15-second rest period. The raw EEG signals were recorded at a sampling rate of 1000Hz, downsampled to 200Hz, and then filtered using a bandpass filter from 0 to 75Hz.

[0043] (2) DEAP, another public dataset collected and released by Kolestra et al. for emotion computing. It collected the physiological electrical signals of 32 subjects. The EEG signals were recorded by a 40-channel electrode cap, including 32 channels of EEG signals and 8 channels of peripheral physiological signals. Each subject's experiment included 40 emotional arousal videos. After watching the videos, the subjects were asked to rate the valence, arousal, dominance, and liking of the self-assessment manikins (SAM) from 1 to 9. The raw EEG signals were recorded at a sampling rate of 512 Hz, and after bandpass filtering from 4 to 45 Hz, the sampling rate was reduced to 128 Hz, and then the data was divided into 60 seconds of experimental data and 3 seconds of baseline data.

[0044] like Figure 1 and Figure 2 As shown, the implementation steps of the embodiments of the present invention are as follows:

[0045] Step 1: Preprocess the EEG signal and extract the differential entropy features of the EEG according to frequency bands. The specific steps are as follows:

[0046] For a sequence of EEG signals x over a period of time, its differential entropy characteristic is defined as:

[0047] h(x)=-∫f(x)log[f(x)]dx (6)

[0048] Where f(x) is the probability density function of the EEG signal, and after bandpass filtering, the time series of the EEG signal follows a Gaussian distribution N(μ,σ). 2 Therefore, formula (6) can be written as:

[0049]

[0050] According to formula (7), the EEG differential entropy feature of each sample can be calculated;

[0051] Step 2: Based on the EEG differential entropy features obtained in Step 1, select the source domain for each target domain. The specific steps are as follows:

[0052] Step 2-1: For the EEG signal of each target domain, calculate the differential entropy feature using formula (7) and divide it into labeled target domain data. and unlabeled target domain data

[0053] Step 2-2: For each source domain A Sp Let p = 1, 2, 3, ..., n, and train each group to obtain a classifier, a support vector machine (SVM). Then, ... Perform classification tests and rank the SVMs based on their classification accuracy.

[0054] Steps 2-3: Based on the sorting results, obtain the transferable weights of different source domains, and based on these weights, remove i source domains and use the remaining data as suitable source domains for subsequent migration.

[0055] Step 3: Transform the EEG signal features into a manifold space to further extract manifold features. The specific steps are as follows:

[0056] Step 3-1: Introduce the Geodesic Flow Kernel (GFK) method to transform the original EEG differential entropy features into the Grassmann manifold G, completing the manifold feature transformation. The definition of the Geodesic Flow Kernel (GFK) is as follows:

[0057]

[0058] Where, x i and x j Let z be a d-dimensional eigenvector. i ,z i Represents the transformed feature z i and z j The inner product between them, the positive semidefinite matrix G∈R d×d It can be calculated using singular value decomposition.

[0059] Step 3-2: Based on the solved positive semi-definite matrix G, the transformed eigenvalue z can be obtained using the formula... Calculations were performed to obtain the EEG manifold characteristics.

[0060] Step 4: After obtaining the EEG manifold features, the features are dimensionality-reduced using the improved maximum correlation and minimum redundancy method to select more representative features. The specific steps are as follows:

[0061] Step 4-1: Based on the EEG waveform feature x of the source domain data and its corresponding label category c, calculate the correlation D between the feature and the label, and the correlation R between the features:

[0062]

[0063]

[0064] Where I(·,·) is the mutual information size, and S is the feature set.

[0065] Step 4-2: Unlike using formula (11) as the evaluation function to guide feature subset selection for dimensionality reduction, a new evaluation function formula (12) is defined by introducing an indicator vector β and a parameter k.

[0066] maxφ(D,R)=DR (11)

[0067]

[0068] Where k is the final feature dimension, β = [β1β2...β] n ], where n is the number of features in the original feature set, β i The closer β is to 1, the more important the feature in that dimension is. i A value of 0 indicates that the i-th feature was not selected.

[0069] Step 4-3: For the objective function (12) in the previous step, use the incremental search method to maximize a quadratic function, solve for the indicator vector β and the final feature dimension k, and select a new source domain feature dataset F′ from the source domain and target domain data according to the above parameters. S and the target domain feature dataset F′ T .

[0070] Step 5: After obtaining the feature datasets of the new target domain and source domain, dynamically align their distributions, adapting them to both marginal distributions and conditional probability distributions, ultimately learning a domain-invariant classifier f. The specific steps are as follows:

[0071] Step 5-1: Perform dynamic distribution alignment Defined as:

[0072]

[0073] Where μ∈[0,1], it measures the importance of marginal distributions and conditional probability distributions. D f (P s ,P t ) represents the marginal distribution difference, D f (c) (Q s Qt ) represents the difference in conditional probability distributions.

[0074] Step 5-2: Introduce the MMD (maximum mean discrepancy) distance to calculate the above differences. The MMD distance between different distributions p and q is defined as:

[0075]

[0076] In the formula, HK is the reproducing kernel Hilbert space (RKHS) spanned by the feature map φ(·), and E(·) is the mean of the embedded samples.

[0077] Step 5-3: Finally, based on the principle of structural risk minimization (SRM), a domain-invariant classifier f can be expressed as:

[0078]

[0079] In the formula, the first two terms represent the loss of the source domain data. For dynamic distribution alignment, R f (D s D t ) represents the Laplacian regularization term, and λ and ρ are the corresponding regularization parameters.

[0080] Step 6: Based on the classifier f obtained in the previous step, the predicted label and accuracy of each source domain for the target domain can be obtained. The recognition results of multiple classifiers are then weighted and fused to obtain the final classification label and recognition accuracy. The specific steps are as follows:

[0081] After obtaining the recognition accuracy of multiple sets of target domain data for multiple sets of source domain data, the recognition accuracy of each classifier is used as a weight to perform weighted fusion on the final predicted label. The weighted fusion calculation method for each sample is as shown in formula (16):

[0082]

[0083] Where n is the index of the source domain, and w is the weight. This indicates that the sample's preliminary result from n source domains is the sum of the weights of class c. The category label corresponding to the largest sum of weights is the final identification label result.

[0084] This invention uses the SEED and DEAP datasets, with 15 subjects in each dataset. One subject is selected as the target domain, and the remaining 14 as the source domain. The experiment is repeated 15 times to ensure that each subject is included as the target domain. The experimental results are as follows: Figure 3 , Figure 4 and Figure 5 As shown. Figure 3 The curves showing the change in recognition accuracy for each subject on the SEED dataset as the number of source domains increases are presented. It can be seen that the accuracy begins to rise when the number of source domains increases from 1 to 7, and the accuracy curve begins to stabilize or even declines when the number of source domains increases from 7 to 14. This indicates that there are some poor-quality source domains, which lead to negative transfer. Therefore, choosing 7 source domains can effectively remove bad source domains. Figure 4 , Figure 5 The paper presents a comparison of the recognition accuracy of the proposed method and various domain adaptive methods on SEED and DEAP for each subject. It can be seen that the proposed method achieves the best recognition accuracy on the vast majority of subjects, demonstrating the effectiveness of the proposed multi-source manifold EEG feature domain adaptive method.

[0085] The embodiments described above are merely preferred examples of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A multi-source manifold embedding feature selection method based on EEG mutual information, characterized in that, The main steps include: Step 1: Acquire EEG signals and preprocess them; Step 2: Filter the data from multiple source domains and select high-quality source domains; Step 3: Transform the EEG signal features into a manifold space to further extract manifold features; Step 4: Perform correlation and redundancy analysis on the electroencephalogram (EEG) features to reduce feature dimensionality; Step 4-1: Based on the EEG waveform feature x of the source domain data and its corresponding label category c, calculate the correlation D between the feature and the label, and the correlation R between the features: Where I(·,·) is the mutual information size, and S is the feature set; Step 4-2: Unlike using formula (3) as the evaluation function to guide feature subset selection for dimensionality reduction, a new evaluation function formula (4) is defined by introducing an indicator vector β and a parameter k. maxφ(D,R)=DR (3) Where k is the final feature dimension, β = [β1β2...β] n ], where n is the number of features in the original feature set, β i The closer β is to 1, the more important the feature in that dimension is. i A value of 0 indicates that the i-th feature was not selected; Step 4-3: For the objective function (5) in the previous step, use the incremental search method to maximize a quadratic function, solve for the indicator vector β and the final feature dimension k, and select a new source domain feature dataset F from the source domain and target domain data according to the above parameters. S 'and target domain feature dataset F T '; Step 5: Transfer the reduced manifold features to learn the final classifier f for label prediction. In the formula, For dynamic distribution alignment, R f (D s D t ) represents the Laplacian regularization term, and λ and ρ are the corresponding regularization parameters; Step 6: Based on the classifier results, perform weighted fusion of the multiple recognition results for the target domain to obtain the final classification result. The weighted fusion method is as follows: After obtaining the recognition accuracy of multiple sets of target domain data for multiple sets of source domain data, the recognition accuracy of each classifier is used as a weight to perform weighted fusion on the final predicted label. The weighted fusion calculation method for each sample is as shown in formula (6): Where n is the index of the source domain, and w is the weight. This indicates that the predicted result of the sample from n source domains is the sum of the weights of class c. The category label corresponding to the largest sum of weights is the final identification label result.

2. The multi-source manifold embedding feature selection method based on EEG mutual information according to claim 1, characterized in that, The preprocessing method in step 1 is to extract EEG differential entropy features according to frequency bands.

3. The multi-source manifold embedding feature selection method based on EEG mutual information according to claim 2, characterized in that, The screening method in step 2 is as follows: When selecting source domains, a classifier is pre-trained using data from each source domain and then applied to a small number of target domain data with existing labels for selection.

4. The multi-source manifold embedding feature selection method based on EEG mutual information according to claim 2, characterized in that, Step 2 specifically includes the following steps: Step 2-1: For the differential entropy features of the EEG signal in each target domain, divide it into labeled target domain data. and unlabeled target domain data Step 2-2: For each source domain A Sp For each p = 1, 2, 3, ..., n, a support vector machine (SVM) classifier is obtained through training. Then, the classifier is... Perform classification tests and rank the SVMs based on their classification accuracy. Steps 2-3: Based on the ranking results, select the data trained by the classifiers with the highest accuracy as high-quality source domains for subsequent transfer learning.

Images