Multi-fractal feature-based electroencephalogram emotion classification method and system

Abstract

The invention discloses an electroencephalogram emotion classification method and system based on multi-fractal features. According to the method, a multi-fractal feature extraction method is adopted, the Hurst index and the singular index of the electroencephalogram data are extracted, the features can more comprehensively reflect local dynamic changes of signals, the robustness of a classification model is improved, the capturing capacity of instantaneous emotion changes is enhanced, and therefore the emotion classification performance is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electroencephalogram (EEG) signal processing and emotion recognition, and in particular to an EEG emotion classification method and system based on multi-fractal features. Background Art

[0002] Electroencephalography (EEG) is a technique that records the brain's electrical activity and can capture the brain's response to different stimuli, including emotional changes. Emotional states are closely related to the brain's electrophysiological activity, so EEG data analysis has important applications in emotion recognition. Despite this, current emotion recognition technologies still have certain limitations. Most of these techniques rely on traditional signal processing methods, such as power spectral density analysis and linear feature extraction, which may not be able to effectively capture the complex patterns and dynamic changes in EEG signals.

[0003] In addition, EEG signals themselves are highly nonlinear and dynamic, which makes it difficult for traditional linear analysis methods to fully utilize the rich information they contain. In addition, the high dimensionality and background noise of EEG data also pose challenges to the accuracy and reliability of emotion classification. Therefore, developing a new method that can more comprehensively analyze the complex patterns in EEG signals is crucial to improving the performance of emotion recognition.

[0004] The application of fractal theory in EEG signal analysis is based on its unique ability to describe the complexity and self-similarity of signals. Fractal features, such as fractal dimension, provide a way to quantify the complexity of EEG signals. This complexity reflects the dynamic changes and nonlinear characteristics of brain activity, which are closely related to changes in emotional states. Traditional linear features may not be able to fully capture this complexity, while fractal features can reveal hidden patterns and structures in EEG signals.

[0005] In the context of emotion classification, fractal features can be used to improve recognition accuracy. Emotional states (such as happiness, sadness, neutrality, and anger) are associated with specific patterns of electrical activity in the brain, which may appear as unique features in fractal analysis. Therefore, by analyzing the fractal features of EEG signals, different emotional states can be distinguished more effectively, thereby improving the performance of emotion classification systems. However, existing EEG emotion classification methods rely on traditional fractal feature extraction (such as fractal dimension and box counting method). Although they have achieved certain results, they have limitations in feature selection, classification accuracy, and sensitivity to dynamic changes. Summary of the invention

[0006] In view of this, the present invention provides an EEG emotion classification method based on multi-fractal features, which can accurately and effectively perform emotion recognition.

[0007] The EEG emotion classification method based on multi-fractal features of the present invention comprises:

[0008] The Hurst index and quality index of EEG data were extracted by multifractal analysis; the quality index was derived to obtain the singular index;

[0009] The Hurst index and singularity index of EEG data are used as fractal features to classify EEG emotions.

[0010] Furthermore, the multifractal analysis method is used to extract the Hurst index and singular index of the EEG data, specifically:

[0011] S1, for the collected EEG data sequence {x k}, calculate the sequence {x k The mean of

[0012]

[0013] Where N is the length of the EEG data sequence;

[0014] S2, calculate the cumulative deviation of the i-th EEG data:

[0015]

[0016] S3, divide the cumulative deviation sequence {Y(i)} into several sub-intervals of equal length;

[0017] S4, for each sub-interval, the points in the sub-interval are fitted with a k-order polynomial using the least squares method:

[0018] y v (m) = a1m k +a2m k-1 ++a k m+a k+1

[0019] Where, m = 1, 2, ..., s, s is the total number of data in the sub-interval;

[0020] S5: Calculate the mean square error F of the sample data for each sample interval. 2 (s,v);

[0021] S6, to F 2 (s,v) takes the average value and obtains the q-wave function F q (s):

[0022]

[0023] Where q is any non-zero real number;

[0024] S7, plot “ln[F q (s)]-lns” function relationship diagram, the slope in the function relationship diagram is the Hurst exponent H(q);

[0025] S8, calculate the quality index τ(q):

[0026] τ(q)=qH(q)-1

[0027] S9, calculate the singular index α:

[0028]

[0029] Furthermore, in S3, the accumulated deviation sequence Y(i) is divided into N S subintervals, where If s is not divisible by N, shift the sequence backward until the rest of the sequence is included, and then divide the shifted sequence into N S subintervals; finally we get 2N S Small intervals of equal length.

[0030] Furthermore, in S5,

[0031] For the interval v=1,2,,N S , calculate the mean square error F 2 (s,v):

[0032]

[0033] For the interval v = N s +1,N s +2,,2N s , calculate the mean square error F 2 (s,v):

[0034]

[0035] Furthermore, the multifractal detrended fluctuation analysis method is a multifractal detrended fluctuation analysis method.

[0036] Furthermore, the classifier used for EEG emotion classification is decision tree, support vector machine, logistic regression or random forest.

[0037] Further, constructing a sample set, training the classifier based on the sample set, and using the trained classifier to complete EEG emotion classification;

[0038] The sample set is constructed in the following ways:

[0039] S01, collect EEG data under different emotions;

[0040] S02, down-sample the EEG data of S01;

[0041] S03, using independent component analysis algorithm to remove eye contact signals from EEG data;

[0042] S04, segmenting the EEG signals so that they correspond to different emotional segments of the subject, and removing the EEG data corresponding to the subject's preparation time;

[0043] S05, extracting EEG samples from the EEG signals segmented in S04 to form a sample set.

[0044] Furthermore, in S01, the EEG data of the DEAP dataset, the SEED dataset, the SEED-IV dataset, or the SEED-V dataset is directly used.

[0045] The present invention also provides an EEG emotion classification system based on multi-fractal features, comprising: a signal acquisition module, a feature extraction module and a classifier;

[0046] Wherein, the signal acquisition module is used to collect EEG data;

[0047] The feature extraction module uses the above method to extract the Hurst index and singular index of the EEG data;

[0048] The classifier takes the Hurst index and singular index of the EEG data as input and outputs the EEG emotion classification result using the above method.

[0049] Beneficial effects:

[0050] (1) The present invention adopts a multi-fractal feature extraction method to extract the Hurst exponent and singular exponent of EEG data. This feature can more comprehensively reflect the local dynamic changes of the signal, improve the robustness of the classification model, and enhance the ability to capture instantaneous emotional changes, thereby effectively improving the performance of emotion classification.

[0051] (2) The Hurst exponent and singular exponent of EEG signals are extracted by using the multifractal detrended fluctuation analysis method. This method increases the ability to remove signal trends and is more effective in processing non-stationary signals than the multifractal analysis method.

[0052] (3) Support vector machine is used as the classifier for sentiment classification. Support vector machine has good robustness to high-dimensional data and noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 The figure is a flow chart of the method of the present invention.

[0054] Figure 2 This is the confusion matrix of the experimental results. DETAILED DESCRIPTION

[0055] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0056] The present invention provides a method and system for classifying EEG emotions based on multi-fractal features.

[0057] EEG data is a complex data with nonlinear and fractal characteristics. The fractal dimension can characterize the morphological characteristics of EEG signals and reflect the complexity of its shape with non-integer dimensions. It can quantify the irregularity and complexity of EEG data well.

[0058] The present invention uses a multi-fractal analysis method to extract the Hurst index and singular index of EEG data, takes the Hurst index and singular index of EEG data as fractal features, and combines a classifier to achieve EEG emotion classification. The Hurst index is used to measure the long-term memory characteristics of a time series, and its value is usually between 0 and 1; the Hurst index can be used to study the stability and complexity of brain activity and help distinguish different brain states (such as relaxation, attention, anxiety, etc.). The singular index can reflect the local complexity and self-similarity of the signal. The high or low singular index can indicate the complexity of the signal and help analyze noise, emergencies or other important features in the signal. The present invention uses the Hurst index and singular index as fractal features to quickly and accurately achieve EEG emotion classification.

[0059] The specific method for obtaining the Hurst index and singular index of EEG data is as follows:

[0060] For an EEG signal sequence {x k}, where k = 1, 2, ..., N, the calculation steps of multifractal analysis are as follows:

[0061] S1, calculate the sequence sample {x k The mean of

[0062]

[0063] S2, calculate the cumulative deviation of the sample signal:

[0064] Where i=1,2,…,N.

[0065] S3, divide the cumulative deviation sequence {Y(i)} calculated in S2 into N S Small intervals, among which

[0066]

[0067] If s is not divisible by N, there will be a section of data in Y(i) that is not used. In order to make full use of the data samples without causing data loss, it is necessary to reuse the divided data samples, move the sequence backward until all the unused data in the sequence is included, and then divide it into N again. S small intervals, and finally get 2N S In this way, all the data of the data sample can be fully utilized to achieve the best effect.

[0068] The following are two examples of data lengths that are divisible:

[0069] Example 1: N=120, s=40

[0070] 1) Calculate N S :

[0071]

[0072] This means we can divide the sequence into 3 intervals of length 40.

[0073] 2) Initial division:

[0074] The first interval: Y(1) to Y(40)

[0075] The second interval: Y(41) to Y(80)

[0076] The third interval: Y(81) to Y(120)

[0077] Thus, we used all 120 data points and there was no leftover data.

[0078] 3) Repeated division:

[0079] Since there is no remaining data, we do not need to repeat the partitioning.

[0080] 4) Final number of intervals:

[0081] We have obtained 3 intervals, each of length 40.

[0082] Example 2: N=127, s=40

[0083] 1) Calculate N S :

[0084]

[0085] This means that we can divide the sequence into 3 intervals of length 40, but there will be leftover data.

[0086] 2) Initial division:

[0087] The first interval: Y(1) to Y(40)

[0088] The second interval: Y(41) to Y(80)

[0089] The third interval: Y(81) to Y(120)

[0090] Thus, we have used 120 data points, leaving 7 data points unused.

[0091] 3) Process the remaining data:

[0092] Since the remaining 7 data points are not enough to form a complete interval of length 40, we need to reuse the existing data.

[0093] 4) Repeated division:

[0094] Shift the series back 7 data points and divide again:

[0095] The first interval: Y(7) to Y(47)

[0096] The second interval: Y(48) to Y(88)

[0097] The third interval: Y(89) to Y(127)

[0098] Thus, we used 120 data points again and covered the 7 data points that were not used before.

[0099] 5) Final number of intervals:

[0100] We get 6 intervals, each of length 40.

[0101] S4, perform k-order polynomial fitting using the least squares method on the s points in each small interval of equal length obtained in S3:

[0102] y v (m) = a1m k +a2m k-1 ++a k m+a k+1

[0103] Where m = 1, 2, ..., s; k is obtained by fitting the current data.

[0104] S5, calculate the mean square error:

[0105] If the interval v=1,2,,N S , calculate the mean square error F 2 (s,v):

[0106]

[0107] If the interval v = N s +1,N s +2,,2N s , calculate the mean square error F 2 (s,v):

[0108]

[0109] S6, to F 2 (s,v) takes the average value and obtains the q-wave function F q (s):

[0110]

[0111] Where q is any non-zero real number; in multifractal analysis, q is a tuning parameter used to adjust the sensitivity to different signal fluctuation intensities. When q is a positive number, high-amplitude fluctuations will dominate, which can emphasize the violent fluctuation area of ​​the signal. When q is a negative number, low-amplitude fluctuations will be emphasized, which will focus more on the gentle area of ​​the signal. q (s) increases with the increase of s, in a power law relationship, that is, F q (s)∝s h(q) . For each s, there is a corresponding function value F q (s), for ln[F q The slope in the [(s)]-lns function relationship diagram is the generalized Hurst exponent H(q).

[0112] S7, calculate the quality index τ(q) through the Hurst index:

[0113] τ(q)=qH(q)-1

[0114] S8, calculate the singular index α through the quality index τ(q):

[0115]

[0116] The singularity index α reflects the degree of singularity of the interval and is inversely proportional to the singularity.

[0117] The Hurst exponent and singular exponent of EEG signals can also be extracted by using Multifractal Detrended Fluctuation Analysis (MF-DFA). MF-DFA is a new multifractal analysis method that increases the ability to remove signal trends and is more effective in processing non-stationary signals than multifractal analysis methods.

[0118] S9, normalize the extracted fractal features to ensure that they are at the same scale to improve the stability of the model.

[0119] The EEG emotion classifier of the present invention can adopt classifiers such as decision tree, support vector machine, logistic regression, random forest, etc. This embodiment adopts support vector machine (SVM), which has good robustness to high-dimensional data and noise.

[0120] A sample set of EEG signals under different emotions can be constructed and divided into a training set, a validation set, and a test set in a ratio of 60%, 20%, and 20%, wherein the training set has no overlap with the validation set and the test set. The classifier will be trained based on the training set, the trained classifier will be verified based on the validation set, and the classification effect will be tested based on the test set.

[0121] The sample set can be constructed by collecting EEG data under different emotions, and can also be constructed based on existing EEG datasets, such as the DEAP dataset, the SEED dataset, the SEED-IV dataset, or the SEED-V dataset. This embodiment uses the SEED dataset, which is a multimodal emotion database based on physiological signals generated under stimulation induced by movie clip materials. The SEED EEG dataset contains EEG data of 15 people, and the data was collected when they watched movie clips. Movie clips are carefully selected to induce different types of emotions, which are: positive, negative, and neutral.

[0122] The specific method of constructing the sample set is as follows:

[0123] S01, download the SEED dataset from https: / / bcmi.sjtu.edu.cn / ~seed / index.html;

[0124] S02, downsample the EEG data of the downloaded SEED dataset at a sampling rate of 200 Hz;

[0125] S03, using independent component analysis algorithm to remove eye contact signals from EEG data;

[0126] S04, read the signals of 64 EEG channels according to the international 10-20 system standard. The electrode channels of the SEDD dataset are ordered as follows: FP1, FPZ, FP2, AF3, AF4, F7, F5, F3, F1, FZ, F2, F4, F6, F8, FT7, FC5, FC3, FC1, FCZ, FC2, FC4, FC6, FT8, T7, C5, C3, C1, CZ, C2, C4, C6, T8, TP7, CP5, CP3, CP1, CPZ, CP2, CP4, CP6, TP8, P7, P5, P3, P1, PZ, P2, P4, P6, P8, PO7, PO5, PO3, POZ, PO4, PO6, PO8, CB1, O1, OZ, O2, CB2;

[0127] The EEG data of the 3-second (200*3=600 data points) preparation time before the test were removed, and then segmented according to the window length of 30 seconds.

[0128] S05, select EEG data samples to build a sample set

[0129] Select m EEG samples S of subjects from the SEED dataset C×N i, where i ranges from [1,m], and EEG sample S C×N The data dimension of i is C×N, where C is the number of channels and N is the number of sampling points to be processed. The processed data sample set is {S1, S2, ..., S m ×C×t}, where t is the number of movie clips played in one trial.

[0130] The Hurst index and singularity index of the EEG signal of the sample set are used as the input of the classifier, and the emotion label is used as the output of the classifier. The classifier is trained, verified and tested, and the classifier parameters are adjusted to improve the performance. The final classifier is used to realize EEG emotion classification.

[0131] Use the test set to evaluate the classifier and calculate performance indicators such as accuracy, precision, recall, and F1 score. The calculation method is as follows:

[0132] illustrate:

[0133] TP (True Positives): True positives, predicted as positive and actually positive;

[0134] FP (False Positives): False positive examples, predicted as positive examples but actually negative examples;

[0135] FN (false Negatives): False negative examples, predicted as negative examples but actually positive examples;

[0136] TN (True Negatives): True negative examples, predicted as negative examples and actually negative examples.

[0137] (1) Accuracy: The ratio of the number of correct predictions in positive and negative examples to the total number, expressed as:

[0138]

[0139] (2) Precision: The ratio of samples predicted as positive to samples predicted as correct is expressed as:

[0140]

[0141] (3) Recall: Based on actual samples, the proportion of positive samples that are predicted to be correct among the total positive samples is expressed as:

[0142]

[0143] (4) F1-score: The F1-score is the harmonic mean of precision and recall, which can be regarded as a comprehensive consideration of the classifier’s accuracy and sensitivity to the positive class. It is expressed by the formula:

[0144]

[0145] The following is explained in conjunction with specific embodiments:

[0146] The embodiment uses the SEED EEG dataset (https: / / bcmi.sjtu.edu.cn / ~seed / index.html) to classify emotions into three categories: positive 1, neutral 0, and negative -1; the results are shown in Table 1 and Figure 2 shown.

[0147] Table 1

[0148] Accuracy Recall F1 score Negative (-1) 0.90 0.90 0.90 Neutral(0) 0.96 0.96 0.96 Active (1) 0.91 0.91 0.91

[0149] Among them, the precision reflects the proportion of samples predicted by the model as positive in this category. Specifically, the precision of category -1 is 0.90, which means that 90% of the samples predicted as -1 are correct; the precision of category 0 is 0.96, which is the best performance; the precision of category 1 is 0.91, which is also good.

[0150] The recall rate indicates the ability of the model to correctly identify positive samples. Specifically, the recall rate of category -1 is 0.90, which means that 90% of all samples with a true value of -1 are correctly identified; the recall rate of category 0 is 0.96, which is the best performance; the recall rate of category 1 is 0.91, which is also good.

[0151] The F1 value is the harmonic mean of precision and recall, which takes these two indicators into consideration. Specifically, the F1 values ​​of the three categories are between 0.90 and 0.96, indicating that the model performs well in all categories.

[0152] The overall accuracy of the model is 0.92, which means that 92% of the samples are correctly classified, which shows that the performance of the model is relatively good.

[0153] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for classifying EEG emotions based on multi-fractal features, characterized in that: include: The Hurst index and quality index of EEG data were extracted by multifractal analysis; the quality index was derived to obtain the singular index; The Hurst index and singularity index of EEG data are used as fractal features to classify EEG emotions.

2. The method according to claim 1, characterized in that The Hurst index and singular index of EEG data were extracted using multifractal analysis, specifically: S1, for the collected EEG data sequence {x k }, calculate the sequence {x k The mean of Where N is the length of the EEG data sequence; S2, calculate the cumulative deviation of the i-th EEG data: S3, divide the cumulative deviation sequence {Y(i)} into several sub-intervals of equal length; S4, for each sub-interval, the points in the sub-interval are fitted with a k-order polynomial using the least squares method: yv(m)=a1mk+a2mk-1+ … +acm+and+1 Where, m = 1, 2, ..., s, s is the total number of data in the sub-interval; S5: Calculate the mean square error F of the sample data for each sample interval. 2 (s,v); S6, to F 2 (s, v) takes the average value and obtains the q-wave function F q (s): Where q is any non-zero real number; S7, plot "ln[F q (s)]-lns” function relationship diagram, the slope in the function relationship diagram is the Hurst exponent H(q); S8, calculate the quality index τ(q): τ(q)=qH(q)-1 S9, calculate the singular index α:

3. The method according to claim 2, characterized in that In S3, the accumulated deviation sequence Y(i) is divided into N S subintervals, where If s is not divisible by N, shift the sequence backward until the rest of the sequence is included, and then divide the shifted sequence into N S subintervals; finally we get 2N S Small intervals of equal length.

4. The method according to claim 3, characterized in that In S5, For the interval v=1,2, … ,NS, calculate the mean square error F2(s,v): For the interval v=Ns+1,Ns+2, … ,2Ns, calculate the mean square error F2(s,v):

5. The method according to claim 1, characterized in that The multifractal detrended fluctuation analysis method is a multifractal detrended fluctuation analysis method.

6. The method according to any one of claims 1 to 5, characterized in that: The classifiers used for EEG emotion classification are decision trees, support vector machines, logistic regression, or random forests.

7. The method according to claim 6, characterized in that Constructing a sample set, training the classifier based on the sample set, and using the trained classifier to complete EEG emotion classification; The sample set is constructed in the following ways: S01, collect EEG data under different emotions; S02, down-sample the EEG data of S01; S03, using independent component analysis algorithm to remove eye contact signals from EEG data; S04, segmenting the EEG signals so that they correspond to different emotional segments of the subject, and removing the EEG data corresponding to the subject's preparation time; S05, extracting EEG samples from the EEG signals segmented in S04 to form a sample set.

8. The method according to claim 7, characterized in that In S01, the EEG data of the DEAP dataset, the SEED dataset, the SEED-IV dataset, or the SEED-V dataset is directly used.

9. An EEG emotion classification system based on multi-fractal features, characterized in that: include: Signal acquisition module, feature extraction module and classifier; Wherein, the signal acquisition module is used to collect EEG data; The feature extraction module is used to extract the Hurst index and singular index of the EEG data; The classifier takes the Hurst index and singular index of the EEG data as input and outputs the EEG emotion classification result; The feature extraction module and the classifier use the classification method as described in any one of claims 1 to 8 to perform feature extraction and EEG emotion classification.

Images