An electroencephalogram emotion classification method, system, device, medium and terminal

Abstract

The application belongs to the technical field of machine learning and intelligent human-computer interaction, and discloses an electroencephalogram emotion classification method, system, device, medium and terminal, the electroencephalogram emotion classification method comprises: carrying out baseline removing pretreatment on electroencephalogram data; the input signal is transformed in units of 5 seconds, the first-order difference value of the time domain signal is calculated, and the first-order difference value is taken as the electroencephalogram feature; the Euclidean distance is used as the standard to select the sample closer to the target domain from the source domain as the actual training sample; the JDA and TCA are used to map the training set and test set samples into the feature space more similar to each other, and the integrated classifier is used for classification evaluation, and the cycle is used to achieve better effect. The application adopts the Euclidean distance for sample screening, increases the TCA feature space conversion in the first cycle, greatly reduces the difference between the source domain and the target domain, shortens the convergence time of the algorithm, and finally realizes the electroencephalogram emotion recognition across subjects.

Description

Technical Field

[0001] This invention belongs to the field of machine learning and intelligent human-computer interaction technology, and in particular relates to a brainwave emotion classification method, system, device, medium and terminal. Background Technology

[0002] Currently, in the field of human-computer interaction, computers and robots must possess emotion processing capabilities to achieve accurate and natural interaction. Methods for emotion recognition vary, ranging from facial images, gestures, and voice signals to other physiological signals. Among these, electroencephalogram (EEG) signals, generated directly by neurons in the brain, are spontaneous and unaffected by the subject's subjective consciousness, offering unique advantages in certain application scenarios. Many scholars have conducted extensive research on EEG emotion recognition.

[0003] Psychological research shows that individuals experience and express emotions in significantly different ways, resulting in substantial differences in the brainwave signals generated by different people for the same emotion. However, most current methods with high accuracy in emotion classification are based on homologous data, meaning that the training and test sets are derived from the same subject or group of subjects, without considering individual specificity. In practical applications, when encountering new individuals, homologous emotion classification methods need to collect a large amount of new data to fully simulate the relationship between the new individual's EEG signals and emotions, and data from different sources must be discarded. In other words, existing classification models rely excessively on the subject and cannot adapt to new individuals, wasting a large amount of data from different sources. Therefore, a new EEG emotion classification method is urgently needed.

[0004] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:

[0005] (1) Most of the current methods with high accuracy in emotion classification are based on the same source data, that is, the training set and the test set data come from the same subject or the same group of subjects, without considering the individual's particularity.

[0006] (2) In practical applications, when encountering new individuals, the homologous emotion classification method needs to collect a large amount of data again to fully simulate the relationship between the EEG signals and emotions of the new individuals, and data from different sources can only be discarded.

[0007] (3) Existing brainwave emotion classification models rely too much on the subjects and cannot adapt to new individuals, resulting in a waste of a large amount of data from different sources.

[0008] The difficulty in solving the above problems and shortcomings lies in how to reduce the differences between individuals or how to find more general EEG features from different source data to obtain more accurate classification results.

[0009] The significance of addressing the above problems and shortcomings is that it allows for the effective identification of the emotional state of new individuals without requiring extensive testing and training to acquire historical data. In practical applications, it is no longer limited to specific known subjects, thus having broader application value. Summary of the Invention

[0010] In view of the problems existing in the prior art, the present invention provides a method, system, device, medium and terminal for classifying brainwave emotions, and particularly relates to a method, system, device, medium and terminal for classifying brainwave emotions based on transfer learning.

[0011] This invention is implemented as follows: a brainwave emotion classification method, the brainwave emotion classification method comprising:

[0012] First, the EEG data undergoes baseline removal preprocessing. Then, the input signal is transformed in 5-second increments to calculate the first-order difference value of the time-domain signal, which is used as the EEG feature. Next, using Euclidean distance as a standard, samples closer to the target domain are selected from the source domain as actual training samples. Finally, JDA and TCA are used to map the training and test set samples to a more similar feature space, and an ensemble classifier is used for classification evaluation. This process is repeated to achieve better results.

[0013] Furthermore, the brainwave emotion classification method includes the following steps:

[0014] Step 1, baseline removal preprocessing: For labeled source domain data and unlabeled target domain data, EEG data in a calm state is recorded as the baseline, while EEG data under video stimulation is recorded as fluctuation data. The fluctuation data is subtracted from the baseline to obtain the relative changes in EEG when a person experiences emotions, which is used as the input data for the experiment; (This step amplifies the fluctuation changes in EEG, improving classification performance.)

[0015] Step 2, EEG Feature Extraction: Using 5s of EEG data as a sample, the input time-domain data for each sample is converted into first-order difference values ​​and used as EEG features to depict the intensity of signal changes in the time domain; (This step reduces the dimensionality of the EEG data, enabling the extraction of variables that are more effective for emotion classification.)

[0016] Step 3, Source Domain Sample Screening: Perform oversampling to balance the number of samples of each type in the source domain; divide the source domain samples into multiple groups of 12 samples each, calculate the standard Euclidean distance between each group and the target domain samples, sort all groups in ascending order of distance, and select the top 85% of source domain samples with smaller distances as the actual training samples; (This step removes some dissimilar samples to overcome the influence of negative transfer.)

[0017] Step 4, Feature Space Transformation: Determine if there are pseudo-labels in the test set (target domain). If there are no pseudo-labels, use the TCA algorithm; if there are pseudo-labels, use the JDA algorithm to find the optimal feature space. After mapping the training samples and test samples to this space, they become more similar. (This step brings the feature distributions between the source and target domain data closer, making the classification model trained in the source domain more suitable for the target domain data.)

[0018] Step 5, Ensemble Classifier Evaluation: Integrate multiple weak SVM classifiers into a strong classifier, evaluate the sentiment classification model, and iterate to achieve optimization. (This ensemble operation can achieve global optimum and better generalization performance; the iterative process continuously adjusts the model, ultimately obtaining a more stable accuracy.)

[0019] Furthermore, in step one, obtaining experimental input data includes:

[0020] In 1 second, X i This represents the raw EEG data at the i-th second (i = 1, 2, ..., n). The first m seconds are EEG data in a calm state, and the subsequent seconds are EEG data under video stimulation. Y i This refers to the input data at the i-th second of the experiment, and the relevant calculation formula is shown below:

[0021]

[0022] Furthermore, in step four, the generation of the pseudo-tags includes:

[0023] Initially, the test set has no labels or pseudo-labels. After feature space transformation using TCA, the classifier predicts a label for each test sample, which serves as the initial pseudo-label. In subsequent loops, the prediction result of the current classifier is used as the pseudo-label for the test set in the next loop.

[0024] Furthermore, in step four, the process of implementing the more similar feature space includes:

[0025] When executing the TCA algorithm, more similar feature spaces are minimized by using the same transformation matrix to minimize the difference in mean between the source and target domain samples;

[0026] By executing the JDA algorithm, a more similar feature space is achieved by minimizing the difference between the means of all samples in the source and target domains using the same transformation matrix, while simultaneously minimizing the sum of the differences between the means of samples in the source and target domains belonging to the same emotion category.

[0027] Furthermore, in step five, the process of achieving the optimization effect through looping includes:

[0028] Repeat steps four and five until the accuracy converges, and use this stable output as the final accuracy.

[0029] Another object of the present invention is to provide an electroencephalogram (EEG) emotion classification system that applies the aforementioned EEG emotion classification method, the EEG emotion classification system comprising:

[0030] The baseline removal preprocessing module is used to record the EEG data in a calm state as the baseline and the EEG data under video stimulation as the fluctuation data for labeled source domain data and unlabeled target domain data. The fluctuation data is subtracted from the baseline to obtain the relative changes in EEG when a person experiences emotions, which is used as the input data for the experiment.

[0031] The EEG feature extraction module is used to take 5s of EEG data as a sample, convert the input time-domain data into first-order difference values ​​for each sample and use them as EEG features to depict the intensity of signal changes in the time domain.

[0032] The source domain sample screening module is used to perform oversampling operations to balance the number of samples of each type in the source domain. The source domain samples are divided into multiple groups of 12 samples each. The standard Euclidean distance between each group and the target domain samples is calculated. All groups are sorted in ascending order of distance, and the top 85% of source domain samples with smaller distances are selected as the actual training samples.

[0033] The feature space transformation module is used to determine whether there are pseudo-labels in the test set, i.e., the target domain. If there are no pseudo-labels, the TCA algorithm is used; if there are pseudo-labels, the JDA algorithm is used to find the optimal feature space, which makes the training samples and test samples more similar after being mapped to this space.

[0034] The integrated classifier evaluation module is used to integrate multiple weak SVM classifiers into a strong classifier to evaluate the emotion classification model and iterate to achieve optimization.

[0035] Another object of the present invention is to provide a computer device comprising a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the following steps:

[0036] First, the EEG data undergoes baseline removal preprocessing. Then, the input signal is transformed in 5-second increments to calculate the first-order difference value of the time-domain signal, which is used as the EEG feature. Next, using Euclidean distance as a standard, samples closer to the target domain are selected from the source domain as actual training samples. Finally, JDA and TCA are used to map the training and test set samples to a more similar feature space, and an ensemble classifier is used for classification evaluation. This process is repeated to achieve better results.

[0037] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the following steps:

[0038] First, the EEG data undergoes baseline removal preprocessing. Then, the input signal is transformed in 5-second increments to calculate the first-order difference value of the time-domain signal, which is used as the EEG feature. Next, using Euclidean distance as a standard, samples closer to the target domain are selected from the source domain as actual training samples. Finally, JDA and TCA are used to map the training and test set samples to a more similar feature space, and an ensemble classifier is used for classification evaluation. This process is repeated to achieve better results.

[0039] Another objective of this invention is to provide an information data processing terminal for implementing the aforementioned brainwave emotion classification system.

[0040] Combining all the above technical solutions, the advantages and positive effects of this invention are as follows: The EEG emotion classification method provided by this invention is based on both sample-based transfer learning and feature-based transfer learning. It uses Euclidean distance for sample selection and improves the original JDA algorithm by adding TCA feature space transformation in the first loop, significantly reducing the difference between the source and target domains, shortening the algorithm's convergence time, and ultimately achieving cross-subject EEG emotion recognition. Compared with traditional machine learning methods, the method described in this invention does not require any labeled target domain samples for training, while simultaneously improving the accuracy of EEG emotion recognition. It has significant theoretical and practical value for research and applications in fields such as human-computer interfaces and wearable devices.

[0041] This invention does not require any labeled EEG data from new individuals for training. It makes full use of EEG data from multiple individuals to predict the emotional state of a new individual. Compared with homologous emotion classification methods, it is easier to find EEG emotional features that are independent of individual differences, has a certain degree of generalization, and has more practical application value.

[0042] This invention utilizes the concept of transfer learning, employing methods such as filtering source domain samples, finding and optimizing the feature space, and ensemble classifiers to bring dissimilar source domain (EEG data from others) and target domain (EEG data from a new individual) samples closer together, obtaining more generalized features and achieving better classification results. The method was experimentally validated on the publicly available EEG dataset DEAP (Database for Emotion Analysis using Physiological Signals). In the Valence dimension, the accuracy of emotion recognition improved from 76.71% to 78.91%; in the Arousal dimension, the accuracy improved from 63.59% to 66.64%. This invention demonstrates improvements in both dimensions and exhibits dimensional generalization. Attached Figure Description

[0043] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a flowchart of the brainwave emotion classification method provided in the embodiments of the present invention.

[0045] Figure 2 This is a schematic diagram of the brainwave emotion classification method provided in the embodiments of the present invention.

[0046] Figure 3 This is a structural block diagram of the EEG emotion classification system provided in an embodiment of the present invention;

[0047] In the diagram: 1. Baseline removal preprocessing module; 2. EEG feature extraction module; 3. Source domain sample screening module; 4. Feature space transformation module; 5. Ensemble classifier evaluation module.

[0048] Figure 4 This is a schematic diagram showing the result of a specific embodiment of the method of the present invention and other methods for performing sentiment binary classification on the Valence dimension.

[0049] Figure 5 This is a schematic diagram showing the result of a specific embodiment of the method of the present invention and other methods for performing sentiment binary classification on the Arousal dimension. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0051] To address the problems existing in the prior art, the present invention provides a method, system, device, medium, and terminal for classifying brainwave emotions. The present invention will be described in detail below with reference to the accompanying drawings.

[0052] like Figure 1 As shown, the brainwave emotion classification method provided in this embodiment of the invention includes the following steps:

[0053] S101, Baseline Removal Preprocessing: For labeled source domain data and unlabeled target domain data, the EEG data in the calm state is recorded as the baseline, while the EEG data under video stimulation is recorded as the fluctuation data. The fluctuation data is subtracted from the baseline to obtain the relative changes in EEG when a person experiences emotions, which is used as the input data for the experiment.

[0054] S102, EEG feature extraction: Using 5s EEG data as a sample, the input time-domain data is converted into first-order difference values ​​for each sample and used as EEG features to depict the intensity of signal changes in the time domain.

[0055] S103, Source Domain Sample Screening: Perform oversampling to balance the number of samples of each type in the source domain; divide the source domain samples into multiple groups of 12 samples each, calculate the standard Euclidean distance between each group and the target domain samples, sort all groups in ascending order of distance, and select the top 85% of source domain samples with smaller distances as the actual training samples.

[0056] S104, Feature Space Transformation: Determine if there are pseudo-labels in the test set, i.e., the target domain. If there are no pseudo-labels, use the TCA algorithm; if there are pseudo-labels, use the JDA algorithm. Find the optimal feature space, where training samples and test samples are more similar after being mapped to this space.

[0057] S105, Ensemble Classifier Evaluation: Integrate multiple SVM weak classifiers into a strong classifier to evaluate the emotion classification model, and repeat the process to achieve optimization.

[0058] The principle diagram of the EEG emotion classification method provided in this embodiment of the invention is as follows: Figure 2 As shown.

[0059] like Figure 3 As shown, the EEG emotion classification system provided in this embodiment of the invention includes:

[0060] Baseline removal preprocessing module 1 is used to record the EEG data in a calm state as the baseline and the EEG data under video stimulation as the fluctuation data for labeled source domain data and unlabeled target domain data. The fluctuation data is subtracted from the baseline to obtain the relative changes in EEG when a person experiences emotions, which is used as the input data for the experiment.

[0061] The EEG feature extraction module 2 is used to take 5s of EEG data as a sample, convert the input time-domain data into first-order difference values ​​for each sample and use them as EEG features to depict the intensity of signal changes in the time domain.

[0062] Source domain sample screening module 3 is used to perform oversampling operations to balance the number of samples of each type in the source domain. The source domain samples are divided into multiple groups of 12 samples each. The standard Euclidean distance between each group and the target domain samples is calculated. All groups are sorted from smallest to largest distance, and the top 85% of source domain samples with smaller distances are selected as the actual training samples.

[0063] Feature space transformation module 4 is used to determine whether there are pseudo-labels in the test set, i.e., the target domain. If there are no pseudo-labels, the TCA algorithm is used; if there are pseudo-labels, the JDA algorithm is used to find the optimal feature space. After the training samples and test samples are mapped to this space, they are more similar.

[0064] The integrated classifier evaluation module 5 is used to integrate multiple SVM (Support Vector Machine) weak classifiers into a strong classifier to evaluate the emotion classification model and iterate to achieve optimization.

[0065] The technical solution of the present invention will be further described below with reference to specific embodiments.

[0066] Example 1

[0067] This invention discloses a brainwave emotion classification method based on transfer learning. The method includes the following steps: First, the brainwave data is preprocessed to remove baselines; then, the input signal is transformed in 5-second increments to calculate the first-order difference value of the time-domain signal, which is used as the brainwave feature; next, using Euclidean distance as a standard, samples closer to the target domain are selected from the source domain as actual training samples; finally, JDA (Transfer Component Analysis) and TCA (Joint Distribution Adaptation) are used to map the training and test set samples to feature spaces that are more similar to each other, and an ensemble classifier is used for classification evaluation, iterating to achieve better results. This invention is both sample-based and feature-based transfer learning. It uses Euclidean distance for sample selection and improves the original JDA algorithm by adding TCA feature space transformation in the first loop, which significantly reduces the difference between the source and target domains, shortens the convergence time of the algorithm, and ultimately achieves cross-subject brainwave emotion recognition. Compared to traditional machine learning methods, the method described in this invention does not require any labeled target domain samples to participate in training, while improving the accuracy of EEG emotion recognition. It has important theoretical significance and practical value for research and application in fields such as human-computer interfaces and wearable devices.

[0068] The EEG emotion classification method based on transfer learning provided in this invention includes:

[0069] The first step is baseline removal preprocessing: For labeled source domain data and unlabeled target domain data, EEG data in a calm state are recorded as the baseline, while EEG data under video stimulation are recorded as fluctuation data. Subtracting the baseline from the fluctuation data yields the relative changes in EEG when a person experiences emotion. This difference better reflects the EEG characteristics of emotion than the original data, and is used as the input data for the experiment.

[0070] The second step is EEG feature extraction: using 5s of EEG data as a sample, the input time-domain data is converted into first-order difference values ​​for each sample, which depict the intensity of signal changes in the time domain and are used as EEG features.

[0071] The third step is source domain sample selection: First, oversampling is performed to balance the number of samples of each type in the source domain. Then, the source domain samples are divided into multiple groups of 12 samples each. The standard Euclidean distance between each group and the target domain samples is calculated. All groups are sorted from smallest to largest distance, and the top 85% of source domain samples with smaller distances are selected as the actual training samples.

[0072] The fourth step is feature space transformation: determine whether the test set (i.e., the target domain) has pseudo-labels. If there are no pseudo-labels, use the TCA algorithm; if there are pseudo-labels, use the JDA algorithm to find the optimal feature space. After the training samples and test samples are mapped to this space, they are more similar.

[0073] The fifth step is to integrate classifier evaluation: integrate multiple weak SVM classifiers into a strong classifier, evaluate the emotion classification model, and repeat the process to achieve optimization.

[0074] The specific implementation process for obtaining experimental input data provided in this embodiment of the invention is as follows: X is calculated in units of 1 second. i This represents the raw EEG data at the i-th second (i = 1, 2, ..., n). The first m seconds are EEG data in a calm state, and the subsequent seconds are EEG data under video stimulation. Y i This refers to the input data at the i-th second of the experiment, and the relevant calculation formula is shown below:

[0075]

[0076] The process of generating pseudo-tags provided in this embodiment of the invention is as follows:

[0077] Initially, the test set has no labels or pseudo-labels. After feature space transformation using TCA, the classifier predicts a label for each test sample, which serves as the initial pseudo-label. In subsequent loops, the current classifier prediction result is used as the pseudo-label for the test set in the next loop.

[0078] The specific implementation process of the more similar feature space provided in the embodiments of the present invention is as follows:

[0079] When executing the TCA algorithm, more similar feature spaces are minimized by using the same transformation matrix to minimize the difference in mean between the source and target domain samples;

[0080] By executing the JDA algorithm, a more similar feature space is achieved by minimizing the difference between the means of all samples in the source and target domains using the same transformation matrix, while simultaneously minimizing the sum of the differences between the means of samples in the source and target domains belonging to the same emotion category.

[0081] The specific implementation process of the loop provided in this embodiment of the invention to achieve the optimization effect is as follows:

[0082] Repeat steps four and five until the accuracy converges, and use this stable output as the final accuracy of this method.

[0083] This invention does not require any labeled EEG data from new individuals for training. It makes full use of EEG data from multiple individuals to predict the emotional state of a new individual. Compared with homologous emotion classification methods, it is easier to find EEG emotional features that are independent of individual differences, has a certain degree of generalization, and has more practical application value.

[0084] This invention utilizes the concept of transfer learning, employing methods such as filtering source domain samples, finding and optimizing the feature space, and ensemble classifiers to bring dissimilar source domain (EEG data from others) and target domain (EEG data from a new individual) samples closer together, obtaining more generalized features and achieving better classification results. The method was experimentally validated on the publicly available EEG dataset DEAP. In the Valence dimension, the accuracy of emotion recognition improved from 76.71% to 78.91%; in the Arousal dimension, the accuracy improved from 63.59% to 66.64%. This invention demonstrates improvements in both dimensions and exhibits dimensional generalization.

[0085] Example 2

[0086] This invention addresses existing problems by providing a brainwave emotion classification method based on transfer learning, such as... Figure 2 As shown, it includes the following steps:

[0087] Step S1: For labeled source domain data and unlabeled target domain data, EEG data in a calm state is recorded as the baseline, while EEG data under video stimulation is recorded as fluctuation data. Subtracting the baseline from the fluctuation data yields the relative changes in EEG when a person experiences emotion. This difference better reflects the EEG characteristics of emotion than the original data, and is used as the input data for the experiment.

[0088] Step S2: EEG feature extraction: Take 5s of EEG data as a sample, then calculate the difference between two consecutive adjacent items in each sample, convert the input time domain data into first-order difference values, and use them as EEG features.

[0089] Step S3: Source Domain Sample Screening: First, perform oversampling to balance the number of samples of each type in the source domain. Then, divide the source domain samples into multiple groups of 12 samples each. Calculate the standard Euclidean distance between each group and the target domain samples. Sort all groups in ascending order of distance and select the top 85% of source domain samples with smaller distances as the actual training samples.

[0090] Step S4: Feature Space Transformation: Determine if the test set (i.e., the target domain) has pseudo-labels. If there are no pseudo-labels, use the TCA algorithm to minimize the difference between the means of the source domain samples and the target domain samples after transformation using the same transformation matrix. If there are pseudo-labels, use the JDA algorithm to minimize not only the difference between the means of all samples in the source domain and the target domain after transformation using the same transformation matrix, but also the sum of the differences between the means of source domain samples and target domain samples belonging to the same emotion category. The mapped training samples and test samples are more similar in the new feature space. Except for the first loop where the test set has no pseudo-labels, the pseudo-labels in each subsequent loop are the prediction results of the test set in the previous loop. Since there are no pseudo-labels in the first loop, the test set labels predicted by the ensemble classifier will be used as pseudo-labels in the next loop.

[0091] Step S5: Ensemble classifier evaluation: Integrate multiple SVM weak classifiers into a strong classifier to evaluate the emotion classification model. Repeat steps S4 and S5 until the classification accuracy reaches a stable output value in multiple iterations, and use this as the final accuracy.

[0092] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0093] The purpose of this invention is to fully utilize the EEG data of others to predict the emotional state of new individuals. First, the first-order temporal difference of the data is extracted as EEG features. Then, Euclidean distance is used to filter out more similar test samples. Next, the TCA and JDA algorithms are used to transform the training and test sets into more similar feature spaces. Finally, multiple SVM classifiers are integrated and iterated to achieve a stable output.

[0094] Step 1: Baseline Removal Preprocessing

[0095] This invention is based on a DEAP preprocessed dataset with a sampling rate of 128 Hz and a frequency range of 4–45 Hz. Each experiment consisted of 3 seconds of baseline data and 60 seconds of data under video stimulation. Physiological signals from 40 channels of the subjects were recorded simultaneously (the first 32 were related to EEG, and the last 8 were related to peripheral physiological signals). A total of 32 subjects participated, each watching 40 videos. Each person's data was recorded in a file containing both data and labels. The data consisted of temporal sampling data of EEG signals (dimension: 40*40*8064), and the labels were the self-evaluation model scores (dimension: 40*4, numerical range: continuous values ​​from 1 to 9). The scores were based on four dimensions: Valence (pleasure), Arousal (arousal), Dominance (dominance), and Liking (liking).

[0096] The data from the first 3 seconds of each experiment were considered as the EEG data of the person in a calm state and recorded as the baseline. The data from the following 60 seconds were the EEG data of the person after being stimulated by the video and recorded as the fluctuation data. Many studies simply discard the data from the first 3 seconds, thus ignoring the information contained in this part of the data. Considering that EEG signals are generated in both a calm state and when emotions are generated, the difference between these signals can reflect the relative changes in EEG when emotions are generated, which better reflects the EEG characteristics of emotions. Therefore, this invention subtracts the baseline from the raw data to obtain the experimental input data.

[0097] In 1 second, X i This represents the raw EEG data at second i = (i = 1, 2, ..., 63). The first 3 seconds are EEG data in a calm state, and the following 60 seconds are EEG data under video stimulation. Y i This refers to the input data at the i-th second of the experiment, and the relevant calculation formula is shown below:

[0098]

[0099] Step 2: EEG Feature Extraction

[0100] The first-order difference in the time domain is the difference between two consecutive adjacent terms of a discrete time-domain signal, typically expressed as the difference between the preceding and following values. Previous research has shown that variations in EEG time series can reflect different emotional states. The purpose of differencing the time-domain signal is to reduce irregular fluctuations between data points, making the fluctuation curve smoother.

[0101] Considering the short duration of emotions, this invention selects 5s of EEG data as a sample, and then converts the time-domain data of each sample in 32 channels into first-order difference values. Thus, each sample can obtain 32-dimensional EEG features, which characterize the intensity of signal changes in the time domain. The specific calculation is as follows.

[0102] Suppose we have a time series data x = [x1, x2, ..., xn] n Calculate the first-order difference, which is the average of the differences between two consecutive adjacent items in the sample. Let fs be the sampling rate and N be the length of the time series x (N = fs / n). s *5), then the first difference D of channel k k The definition is shown in the following formula.

[0103]

[0104] Step S3: Source Domain Sample Screening

[0105] First, an oversampling operation is performed to balance the source domain samples. The main idea is to randomly find the nearest neighbor of each minority class sample, and then randomly select a point on the spatial line between the two as the new sample. Since the test set is 12 samples divided from a 60-second video, in order to facilitate the calculation of standard Euclidean distance, the source domain samples need to be grouped into groups of 12 samples, sorted according to the distance, and the top 85% of samples with the closest distance are selected as the actual training samples.

[0106] Setting X nk Y is the k-th channel data of the n-th sample in each source domain. nk S is the k-th channel data of the n-th sample in each target domain. k Let K be the standard deviation of the k-th channel after merging the source and target domain samples. Let N be the number of samples and K be the number of channels. The standard Euclidean distance for each group of samples is defined as follows.

[0107]

[0108] Step S4: Feature Space Transformation:

[0109] The algorithm determines whether the test set (i.e., the target domain) has pseudo-labels. If no pseudo-labels are found, the Transform-Based Algorithm (TCA) is used to make the source and target domain samples more similar after transformation using the same transformation matrix. TCA is a classic marginal distribution adaptive method, belonging to feature-based transfer learning. Its goal is to obtain a pair of mapped source and target domain data with the minimum mean difference. If pseudo-labels are found, the Judicial Analysis (JDA) algorithm is used to find a new, more similar feature space. JDA is a further enhancement of TCA, based on the pseudo-labels in the target domain. Besides minimizing the mean difference between the mapped source and target domain data, it also minimizes the cumulative sum of the mean differences between the source and target domain data for various sentiment types. Except for the first iteration where the test set has no pseudo-labels, the pseudo-labels in each subsequent iteration are the predictions from the previous iteration's test set. The test set labels predicted by the ensemble classifier after the TCA feature space transformation in the first iteration will be used as the pseudo-labels in the next iteration.

[0110] The original JDA algorithm uses initial pseudo-labels predicted by a traditional simple classifier (such as KNN) without feature space transformation. This results in dissimilarity between the source and target domains, leading to significant errors and requiring multiple iterations to achieve convergence. This invention combines TCA with the JDA algorithm, adding TCA feature space transformation to the simple classifier in the initial iteration. This significantly narrows the gap between the source and target domains, requiring only minor adjustments in subsequent iterations. This reduces the number of iterations before convergence and shortens the overall runtime of the invention.

[0111] Let x be the source domain data and y be the target domain data. This refers to the new data obtained after x is mapped through a transformation matrix, where n is the number of samples in the source domain, m is the number of samples in the target domain, and x k For sample data labeled as the k-th emotion category in the source domain, y k For sample data in the target domain with pseudo-labels of the k-th emotion class, n k Let m be the number of samples in the source domain labeled with the k-th emotion category. k Let be the number of samples in the target domain labeled as the k-th emotion category. Then, the difference of means is defined as follows:

[0112]

[0113]

[0114] The TCA algorithm is min d1, and the JDA algorithm is min d1+d2. By introducing kernel methods and the Lagrange method, which are common in machine learning, we can solve the above equations to obtain the mapped source domain. and target domain data

[0115] Step 5: Evaluation of the ensemble classifier:

[0116] Multiple SVM weak classifiers are integrated into a strong classifier, and this emotion classification method is evaluated and validated on it. Steps four and five are executed iteratively until the classification accuracy converges, and this convergence is taken as the final accuracy.

[0117] This invention conducts an experiment on binary classification of sentiment based on the Valence and Arousal dimensions. The binary classification standard is based on 5. Values ​​greater than 5 are considered positive, and values ​​less than 5 are considered negative. The classification criteria are shown in Table 1.

[0118] Table 1 Classification Criteria

[0119]

[0120] The effects of the method of the present invention will be described in more detail through the following experimental examples. These experimental examples are for illustrative purposes only and are not intended to limit the scope of application of the present invention.

[0121] The technical effects of the present invention will be described in detail below with reference to experiments.

[0122] Experiment 1: Using this invention to perform sentiment binary classification on the Valence dimension

[0123] Valence, also known as valence or pleasure level, is used to measure the degree of pleasure of an emotion, such as hatred and liking. All sample data in the DEAP dataset are selected, and the labels only need to include the valence dimension.

[0124] First, baseline removal preprocessing was performed. After subtracting the baseline data from the first 3 seconds of data from the 60-second video stimulation data, each video contained 60 seconds of EEG data.

[0125] Then, feature extraction is performed. A 60-second video is divided into 12 samples, each lasting 5 seconds, and each 12 samples are assigned the same label. The difference between two consecutive terms in each channel of each sample is calculated, summed, and averaged to obtain the first-order difference value, which is used as the EEG feature. Thus, a person has 40 videos, each containing 12 samples, each sample containing 32 dimensions, and each sample is used for emotion labeling.

[0126] Next, source domain samples were selected. Twelve samples from one person's video were used as the target domain, i.e., the test set, while 372 samples from the same video of 31 other people were used as the source domain. After oversampling to balance the source domain samples, the standard Euclidean distance between each group of 12 samples and the target domain samples was calculated. All groups were sorted according to the distance, and the 312 (85%) source domain sample groups with the closest distance were selected as the actual training samples.

[0127] Then, feature space transformation is performed. In the first execution, the test set has no pseudo-labels, and the TCA algorithm is used for feature space transformation. In subsequent executions, there are pseudo-labels, which are derived from the prediction results of the test set in the previous loop. The JDA algorithm is used to make the mapped training samples and test samples more similar in the new feature space.

[0128] Finally, an ensemble classifier evaluation is performed, integrating multiple weak SVM classifiers into a strong classifier to evaluate the emotion classification model. This process involves iteratively performing feature space transformation and ensemble classifier evaluation until the classification accuracy reaches a stable value across multiple iterations. In this experimental example, with a small test set, stable outputs are obtained for all samples after 10 iterations. Since the 12 samples come from the same video and belong to the same emotion, the prediction results are determined by majority vote to serve as the final predicted emotion state for this video. Each person's video serves as the test set for cross-validation, and the entire process is executed 32*40 times. The average of these iterations yields the final accuracy of this invention.

[0129] Figure 4This comparison focuses on the results of various methods for sentiment binary classification along the Valence dimension. Traditional machine learning methods refer to directly feeding source domain data into an SVM classifier to obtain the average accuracy. Methods without spatial transformation refer to omitting step four in this invention. The traditional machine learning method achieves an accuracy of 76.95%, while this invention improves the accuracy to 78.90%. However, without spatial transformation, the accuracy will decrease. Furthermore, without sample selection (i.e., omitting step three in this invention), the accuracy drops by more than 30% to 41.48% due to interference from dissimilar samples. These results demonstrate the superiority of this invention; each step plays a crucial role, and the applicability of transfer learning methods in this field is proven.

[0130] Experiment 2: Using this invention to perform binary classification of sentiment on the Arousal dimension

[0131] Arousal, also known as arousal level, is used to measure the intensity of emotions, such as boredom and excitement. All sample data in the DEAP dataset are selected, and the labels only need to include the arousal dimension.

[0132] First, baseline removal preprocessing was performed. After subtracting the baseline data from the first 3 seconds of data from the 60-second video stimulation data, each video contained 60 seconds of EEG data.

[0133] Then, feature extraction is performed. A 60-second video clip is divided into 12 samples, each representing a 5-second interval, and each 12 samples are assigned the same label. The difference between consecutive adjacent terms in each channel of each sample is calculated, summed, and averaged to obtain the first-order difference value, which is used as the EEG feature. Next, source domain samples are screened. After oversampling to balance the source domain samples, the samples are grouped into sets of 12. The standard Euclidean distance between each group of source domain samples and the target domain samples is calculated. All groups are sorted according to the distance, and the 312 (85%) source domain sample groups with the closest distance are selected as the actual training samples. Feature space transformation is then performed. In the first execution, the test set has no pseudo-labels, and the TCA algorithm is used for feature space transformation. In subsequent executions, pseudo-labels are generated from the prediction results of the test set in the previous loop. The JDA algorithm is used to make the mapped training and test samples more similar in the new feature space.

[0134] Finally, an ensemble classifier evaluation is performed, integrating multiple weak SVM classifiers into a strong classifier to classify and evaluate the emotion classification model. This process is repeated cyclically to obtain a stable output. Since the 12 samples come from the same video and belong to the same emotion, the prediction results are voted on by majority vote to determine the final predicted emotion state for this video. Each person's video serves as a test set for cross-validation. The entire process is executed 32*40 times, and the average is used to obtain the final accuracy of this invention.

[0135] Figure 5 This comparison examines the results of various methods for binary sentiment classification along the Arousal dimension. Traditional machine learning methods achieve an accuracy of 63.59%, while this invention improves the accuracy to 67.65%. However, the accuracy decreases further without spatial transformation. Furthermore, without sample filtering, interference from dissimilar samples significantly reduces the accuracy to only 34.38%. Although the original source and target domains are less similar along the Arousal dimension than along the Valence dimension, making prediction more challenging, this invention still achieves higher accuracy. These results demonstrate that this invention exhibits strong generalization performance in cross-individual EEG sentiment classification applications.

[0136] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented, in whole or in part, as a computer program product, the computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0137] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A brainwave emotion classification method, characterized in that, The brainwave emotion classification method includes: First, the EEG data undergoes baseline removal preprocessing. Then, the input signal is transformed in 5-second increments to calculate the first-order difference value of the time-domain data, which is used as the EEG feature. Next, Euclidean distance is used to select samples closer to the target domain from the source domain as the actual training set samples. Finally, JDA and TCA are used to map the training set samples and test set samples to a more similar feature space, and an ensemble classifier is used to evaluate the emotion classification model. This process is repeated to achieve better results. The brainwave emotion classification method includes the following steps: Step 1, baseline removal preprocessing: For labeled source domain data and unlabeled target domain data, the EEG data in the calm state is recorded as the baseline, while the EEG data under video stimulation is recorded as the fluctuation data. The fluctuation data is subtracted from the baseline to obtain the relative changes in EEG when a person experiences emotions, which is used as the input data for the experiment. Step 2, EEG feature extraction: Using 5s of EEG data as a sample, the input time-domain data for each sample is converted into first-order difference values ​​and used as EEG features to depict the intensity of signal changes in the time domain. Step 3, Source Domain Sample Screening: Perform oversampling to balance the number of samples of each type in the source domain; divide the source domain samples into multiple groups of 12 samples each, calculate the Euclidean distance between each group and the target domain samples, sort all groups in ascending order of distance, and select the top 85% of source domain samples with smaller distances as the actual training set samples. Step 4, Feature Space Transformation: Determine if there are pseudo-labels in the test set (i.e., the target domain). If there are no pseudo-labels, use the TCA algorithm; if there are pseudo-labels, use the JDA algorithm to find the optimal feature space, where the training set samples and test set samples are more similar after being mapped to this space. Step 5, ensemble classifier evaluation: Integrate multiple SVM weak classifiers into a strong classifier, evaluate the emotion classification model, and repeat the process to achieve optimization. In step one, the experimental input data is obtained, including: In 1 second, This represents the raw EEG data at the i-th second, where i = 1, 2, ..., n. The first m seconds are EEG data from a calm state, and the subsequent seconds are EEG data under video stimulation. That is, the first experiment The input data in seconds is calculated using the following formula: ， ； Step four, the generation of the pseudo-tags, includes: Initially, the test set has no labels or pseudo-labels. After feature space transformation using TCA, the classifier predicts a label for each test set sample, which serves as the initial pseudo-label. In subsequent loops, the prediction result of the current classifier is used as the pseudo-label for the test set in the next loop. Step four, the implementation process of the more similar feature space, includes: When executing the TCA algorithm, more similar feature spaces are minimized by using the same transformation matrix to minimize the difference in mean between the source and target domain samples; By executing the JDA algorithm, a more similar feature space is achieved by minimizing the difference between the means of all samples in the source and target domains using the same transformation matrix, while simultaneously minimizing the sum of the differences between the means of samples in the source and target domains belonging to the same emotion category.

2. The brainwave emotion classification method as described in claim 1, characterized in that, Step five, the process of achieving the optimization effect through the loop, includes: Repeat steps four and five until the accuracy converges, and use this stable output as the final accuracy.

3. A brainwave emotion classification system implementing the brainwave emotion classification method according to any one of claims 1 to 2, characterized in that, The brainwave emotion classification system includes: The baseline removal preprocessing module is used to record the EEG data in a calm state as the baseline and the EEG data under video stimulation as the fluctuation data for labeled source domain data and unlabeled target domain data. The fluctuation data is subtracted from the baseline to obtain the relative changes in EEG when a person experiences emotions, which is used as the input data for the experiment. The EEG feature extraction module is used to take 5s of EEG data as a sample, convert the input time-domain data into first-order difference values ​​for each sample and use them as EEG features to depict the intensity of signal changes in the time domain. The source domain sample filtering module is used to perform oversampling operations to balance the number of samples of each type in the source domain. The source domain samples are divided into multiple groups of 12 samples each. The Euclidean distance between each group and the target domain samples is calculated. All groups are sorted in ascending order of distance, and the top 85% of source domain samples with smaller distances are selected as the actual training set samples. The feature space transformation module is used to determine whether there are pseudo-labels in the test set, i.e., the target domain. If there are no pseudo-labels, the TCA algorithm is used; if there are pseudo-labels, the JDA algorithm is used to find the optimal feature space, which makes the training set samples and test set samples more similar after being mapped to this space. The integrated classifier evaluation module is used to integrate multiple weak SVM classifiers into a strong classifier to evaluate the emotion classification model and iterate to achieve optimization.

4. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the EEG emotion classification method as described in any one of claims 1 to 2.

5. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the EEG emotion classification method as described in any one of claims 1 to 2.

6. An information data processing terminal, characterized in that, The information data processing terminal is used to implement the EEG emotion classification system as described in claim 3.

Images