Emotion recognition method based on empirical mode decomposition and improved multi-scale fuzzy entropy

Abstract

The invention discloses an emotion recognition method based on empirical mode decomposition and improved multi-scale fuzzy entropy, and belongs to the field of emotion information recognition. Existing emotion electroencephalogram recognition research still has the problems that important information is lost or removed incompletely due to a jaw artifact removal method, the real-time performance and accuracy of emotion recognition are affected by the multi-channel crosstalk problem, and an emotion classification method has limitation and is low in generalization performance. The invention discloses an emotion recognition method based on empirical mode decomposition and improved multi-scale fuzzy entropy. The emotion recognition method comprises the steps that a DEAP data set is selected and imported into EEGlab for preprocessing; using an empirical mode to decompose IMF of each frequency band; using the improved multi-scale fuzzy entropy to calculate the fuzzy entropy of the IMF of each frequency band as a feature vector; obtaining an optimal feature vector by using a maximum mutual information coefficient method; and inputting the optimal feature vector into an SVM, inputting a sample into a high-dimensional space, finding an optimal classification surface, and realizing sample separation. Feature extraction and feature selection modes adopted by the method are more beneficial to emotion classification.

Description

Technical Field

[0001] The invention relates to a method for identifying an emotion signal, and in particular to an emotion recognition method based on empirical mode decomposition and improved multi-scale fuzzy entropy. Background Art

[0002] Human emotions play a core role in decision-making, social interaction, and diagnosis of mental illnesses such as depression. Since machines cannot understand personal feelings, emotions need to be analyzed and calculated. The main purpose of emotional computing is to enable artificial intelligence to have the ability to recognize, analyze and express, establish a friendly and healthy human-computer environment, and improve the intelligence level of computers, so that such interactions are more accessible, usable, and effective. It can be seen that emotional computing is an important part of the field of human-computer interaction and has become a hot topic in human-computer interaction in recent years.

[0003] BCI (Brain Computer Interface) systems can be applied to various industries including healthcare and entertainment. The focus is on potential medical applications in assistive technology and disease classification, such as brain-interface-based prostheses, brain-interface-based wheelchair control, automatic disease diagnosis, assistive cursor control, and wearable devices.

[0004] Nowadays, dealing with big data has become more and more common, especially in the medical field. Due to multi-hour and multi-channel acquisition, the data has become extremely redundant and huge, such as EEG signal acquisition. Therefore, one of the basic goals of feature extraction is dimensionality reduction and data compression. Therefore, this will allow people to represent their data with a smaller subset of features, which will help to effectively use machine learning (ML) and artificial intelligence (AI) algorithms for feature classification and disease diagnosis applications. And useful features should more accurately represent the underlying signal. The problem of emotion recognition based on EEG signals is essentially a pattern recognition problem, and the core part is the learning and classification of patterns. At present, many scholars have proposed emotion classification models based on machine learning, which have good nonlinear approximation capabilities. In recent years, with the rapid development of computer technology, deep learning algorithms have become more and more popular, and they have demonstrated extremely powerful computing and learning capabilities in the field of emotion recognition. However, there are still some urgent problems to be solved in the current research on emotion EEG recognition: (1) The EEG signal itself has a low amplitude, and the non-invasive EEG acquisition equipment makes it susceptible to interference from eye contact artifacts and noise during the acquisition process, while traditional artifact removal methods can easily lead to the loss of important information or incomplete removal; (2) The multi-channel crosstalk problem can lead to the "dimensionality disaster" or feature redundancy of EEG features, affecting the real-time and accuracy of emotion recognition. When extracting emotion-related information, the EEG channel selection should be optimized to select channels with high emotion weights; (3) When classifying emotions, traditional single algorithm classifiers have limitations and low generalization performance, and research on improving classifier performance needs to be strengthened.

[0005] Empirical mode decomposition (EMD) can be used to process nonlinear features such as EEG signals. EMD and its extended methods have high classification performance for biomedical signals of different applications. It decomposes the signal into multiple intrinsic mode functions (IMFs) without making any prior assumptions or defining standards. However, EMD can also have poor decomposition effects due to various problems, such as mode aliasing, endpoint effects, criteria for intrinsic mode functions, over-envelope, under-envelope phenomena, etc. There is still room for improvement in the ability of EMD to decompose EEG signals.

[0006] Entropy has been widely studied as an effective measure for evaluating the dynamic complexity of signals. EEG is a complex nonlinear non-stationary signal encoding with rich dynamics. As a common type of measure in EEG signal analysis, several entropy-based measures are used in different applications, including approximate entropy and sample entropy. Puthakattil and Joeph used signal entropy to distinguish between EEGs of depression and control groups. In another study of theirs, the ApEn value of EEGs of depression was lower than that of healthy controls, indicating that the complexity and predictability of EEGs of depression were lower. P. Samal Cukic used HFD and sample entropy to analyze the EEG signals of depression patients and healthy groups. The results showed that depression patients had higher HFD and sample entropy values, the complexity of the frontal and central parietal regions of depression patients changed, and the complexity of the frontal and parietal regions of depression patients at different stages was different. The study showed that the sample entropy value of the depression group was greater than that of the normal group, indicating that the EEG signal complexity of the depression group was higher. Mohammadi et al. used nonlinear features such as fuzzy entropy, KFD and fuzzy fractal dimension (FFD) to analyze the resting EEG signals of 60 subjects with different levels of depression. The results showed that KFD had a good predictive ability for depression levels, and in almost all experiments, the proposed fuzzy classifier showed more significant superiority than the support vector machine (SVM). Sample entropy and fuzzy measure entropy were used to analyze the EEG signals of patients with different levels of depression. Compared with sample entropy, fuzzy measure entropy showed more significant differences in distinguishing the EEG signals of the two groups of subjects.

[0007] Emotion recognition based on EEG signals has become the mainstream direction in the field of emotion recognition. With the in-depth study of human emotions and the rapid development of electrode technology in EEG sampling, EEG signals have been widely used in various aspects:

[0008] (1) Detection of mental illness. Depression and anxiety disorders are often studied using non-invasive brain-computer interface (BCI) technology based on EEG signals. Minkowski et al. found that the asymmetry of EEG signals originates from the posterior part of the brain. The results of the study can be used to identify and help verify the diagnosis of patients with depression and anxiety. Patients can be measured and analyzed using non-invasive equipment to determine their condition.

[0009] (2) Help patients with central nervous system injuries and epilepsy. Combining epidural stimulation with rehabilitation training can help patients with spinal cord injuries achieve assisted walking when the stimulator is turned on. The first clinical trial of a patient with complete motor spinal cord injury has been completed. The electrical pulse signal emitted by the pulse stimulator stimulates the left vagus nerve, regulates the excitability of the cerebral cortex, and reduces the patient's disease attack rate.

[0010] The study of emotion classification of EEG signals is a new direction in the field of artificial intelligence in the future. How to effectively extract and optimize emotion-related features in EEG signals and improve the accuracy of emotion recognition is a key proposition for emotion recognition. This paper will conduct a detailed study on the application of EEG signals in emotion recognition. Summary of the invention

[0011] The purpose of the present invention is to solve the problems in the existing research on emotion EEG recognition, such as the artifact removal method leading to the loss or incomplete removal of important information, the multi-channel crosstalk problem affecting the real-time and accuracy of emotion recognition, and the limitation of emotion classification method and low generalization performance. An emotion recognition method based on empirical mode decomposition and improved multi-scale fuzzy entropy is proposed.

[0012] The above purpose is achieved through the following technical solutions:

[0013] The emotion recognition method based on empirical mode decomposition and improved multi-scale fuzzy entropy is implemented by the following steps:

[0014] Step 1: Select the DEAP dataset and the six channels of the frontal lobe and the two channels of the temporal lobe related to emotion classification as the raw data, and import them into EEGlab for preprocessing. The preprocessing methods include: data downsampling, removal of EOG artifacts, 4.0-45Hz bandpass filtering, data averaging common reference, and event marking;

[0015] Step 2: Use empirical mode decomposition to decompose the preprocessed data under each stimulus to decompose the IMF of each frequency band;

[0016] Step 3: Calculate the fuzzy entropy of the IMF of each frequency band using the improved multi-scale fuzzy entropy, and use the calculated fuzzy entropy as the feature vector;

[0017] Step 4: Use the maximum mutual information coefficient to calculate the similarity between feature vectors, sort the maximum mutual information coefficient values ​​between features in descending order, and find the feature vector with the highest ranking that represents the most classification characteristics as the optimal feature vector;

[0018] Step 5: Input the optimal feature vector into SVM, input the sample into high-dimensional space, find the optimal classification surface, and achieve sample separation.

[0019] Furthermore, the process of using empirical mode decomposition to decompose the pre-processed data of each subject under each stimulation segment to decompose the IMF of each frequency band is specifically as follows:

[0020] For a given non-stationary time series x(t), EMD adaptively decomposes the time series into a set of IMFs and a residual, where the IMFs have the same extreme values ​​and number of zero crossings and are symmetric with respect to the local zero mean; EMD decomposes the time series x(t) into a set of IMFs and residuals, as shown in the equation:

[0021]

[0022] Furthermore, the process of calculating the improved multi-scale fuzzy entropy described in step 3 and using the calculated fuzzy entropy as the feature vector is specifically as follows:

[0023] Fuzzy entropy,Fuzzy entropy is an indicator used to measure the uncertainty in fuzzy sets. Its similarity measurement formula uses an exponential function as a fuzzy function. The continuity of the exponential function makes the fuzzy entropy smooth. In addition, fuzzy entropy introduces the concept of fuzzy sets to measure the similarity of two vectors.

[0024] Assume that fuzzy entropy is defined as follows:

[0025] For a given N-dimensional time series [u(1),u(2),…,u(i+m-1)], define the phase space dimension m (m≤n-2) and reconstruct the phase space:

[0026] X(i)=[u(i),u(i+1),…u(i+m-1)]-u0(i), i=1,2,…N-m+1

[0027] Among them, u0(i) is the mean, expressed as:

[0028]

[0029] Defining vectors and The distance is the Chebyshev distance, that is, the maximum absolute value of the difference between the numerical values ​​of each element:

[0030]

[0031] Introducing fuzzy membership n measure The similarity of , for 1≤j≤Nm,j≠i is:

[0032]

[0033] The fuzzy similarity function is defined as:

[0034]

[0035] The obtained fuzzy entropy estimate is:

[0036] FuzzyEn(m,n,r,N)=lnΦ m (r)-lnΦ m+1 (r)

[0037] And design an improved fuzzy entropy calculation method: the time coarse-grained method is used to divide the intrinsic mode function after empirical mode decomposition into non-overlapping segments of size τ, and the data of each segment is averaged; for a given time series x of N data points and parameter τ, the coarse-grained time series y(τ) is constructed as follows:

[0038]

[0039] Then, the fuzzy entropy of the time series after time coarse-graining is calculated; and the original Gaussian similarity curve is replaced by an S-type similarity curve, which is expressed as:

[0040] The method of introducing two thresholds α and β (0≤β≤α≤1) divides the membership into three non-overlapping areas, namely complete membership, completely incompatible membership and fuzzy membership; the introduced threshold method is expressed as:

[0041]

[0042] Furthermore, the process of calculating the similarity between feature vectors using the maximum mutual information coefficient described in step 5 is specifically:

[0043] The maximum mutual information coefficient is used to measure the degree of association between two variables, including linear and nonlinear relationships. The maximum mutual information coefficient calculation steps are as follows:

[0044] First, given i and j, the scatter plot composed of XY is gridded with i rows and j columns, and the maximum mutual information value is calculated:

[0045]

[0046] Afterwards, the maximum mutual information value is normalized:

[0047]

[0048] Afterwards, the maximum value of the mutual information under different i and j scales is selected as the MIC value.

[0049] The beneficial effects of the present invention are:

[0050] Emotion plays a core role in a series of intelligent activities such as perception, decision-making, logical reasoning and social interaction, and is an important element in realizing human-computer interaction and machine intelligence. In recent years, with the explosive growth of multimedia data and the rapid development of artificial intelligence, emotional computing and understanding have attracted widespread attention. Emotional computing and understanding aims to give computer systems the ability to recognize, understand, express and adapt to human emotions to establish a harmonious human-computer environment and enable computers to have higher and more comprehensive intelligence. Compared with non-physiological signals, EEG data is more objective, accurate and difficult to disguise due to its spontaneous characteristics.

[0051] The beneficial effects of the present invention are embodied in the following aspects:

[0052] 1. Select 6 frontal lobe and two temporal lobe channels related to emotions to extract EEG signals, and select the middle 16-second data segment to extract features, so as to extract more effective features to the maximum extent possible.

[0053] 2. Use empirical mode decomposition to adaptively decompose EEG data, adaptively decompose complex EEG data into intrinsic mode functions of each frequency, and reflect the trend of superimposed oscillation of EEG signals, so as to more accurately extract emotion-related features.

[0054] 3. Select the improved multi-scale fuzzy entropy as the feature: Based on the original fuzzy entropy, first use the time coarse-graining method to calculate the fuzzy entropy features at different scales; change the original similarity function into an S-shaped curve; introduce the method of two thresholds α and β (0≤β≤α≤1) to divide the membership into three non-overlapping areas, namely complete membership, completely incompatible membership and fuzzy membership. This division makes the understanding of similarity clearer and helps to accurately evaluate the similarity of the system. In addition, the settings of the thresholds α and β are adjustable and can be adjusted according to the actual scenario, which is more in line with the needs of actual problems.

[0055] 4. Calculate the similarity between features and based on the maximum mutual information coefficient, so as to screen out feature sequences that are highly correlated with various emotions, and use feature selection methods to reduce data redundancy and improve classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flow chart of the method of the present invention;

[0057] Figure 2 is a diagram of the introduced α and β threshold method involved in the present invention;

[0058] Figure 3 It is an experimental flow chart involved in the present invention;

[0059] Figure 4 It is a diagram of the arousal and valence model involved in the present invention;

[0060] Figure 5 The present invention relates to a 32-channel electrode distribution diagram for collecting EEG signals;

[0061] Figure 6 It is a schematic diagram of the empirical mode decomposition principle involved in the present invention;

[0062] Figure 7 is the fuzzy entropy value of each component involved in the present invention after calculation;

[0063] Figure 8 It is a similarity curve and a threshold method diagram thereof involved in the present invention;

[0064] Fig. 9 It is a diagram of the feature selection framework involved in the present invention. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Specific implementation method one:

[0067] The emotion recognition method based on empirical mode decomposition and improved multi-scale fuzzy entropy in this embodiment is as follows: Figure 1 As shown, the method is implemented by the following steps:

[0068] Step 1: Select the DEAP dataset and the six channels of the frontal lobe and the two channels of the temporal lobe related to emotion classification as the raw data, and import them into EEGlab for preprocessing. The preprocessing methods include: data downsampling, removal of EOG artifacts, 4.0-45Hz bandpass filtering, data averaging common reference, event marking, etc.

[0069] Step 2: Use empirical mode decomposition to decompose the pre-processed data of each subject under each stimulus to decompose the IMF of each frequency band;

[0070] Step 3: Calculate the fuzzy entropy of the IMF of each frequency band using the improved multi-scale fuzzy entropy, and use the calculated fuzzy entropy as the feature vector;

[0071] Step 4: Use the maximum mutual information coefficient to calculate the similarity between feature vectors, sort the maximum mutual information coefficient values ​​between features in descending order, and find the feature vector with the highest ranking that represents the most classification characteristics as the optimal feature vector;

[0072] Step 5: Input the optimal feature vector into SVM, input the sample into high-dimensional space, find the optimal classification surface, and achieve sample separation. Specific implementation method 2:

[0074] Different from the first specific implementation, the emotion recognition method based on empirical mode decomposition and improved multi-scale fuzzy entropy in this implementation is that the process of using empirical mode decomposition to decompose the pre-processed data of each subject under each stimulus segment and decomposing the IMF of each frequency band is as follows:

[0075] Empirical mode decomposition, since EEG signals are nonlinear and non-stationary, at any given time, EEG signals may produce several different oscillation modes, which may lead to large deviations in the measurement based on fuzzy entropy. The intrinsic mode function of empirical mode decomposition contains local feature information of different time scales, and can also reflect the trend of signal superposition oscillation, so EMD is introduced: it provides an effective tool for detecting information from nonlinear and non-stationary processes because it adopts an adaptive calculation method based on the time scale of local features of data. For a given non-stationary time series x(t), EMD adaptively decomposes the time series into a set of IMFs and a residual, where the IMFs have the same extreme values ​​and number of zero crossings, and are symmetric with respect to the local zero mean; EMD decomposes the time series x(t) into a set of IMFs and residuals, as shown in the equation:

[0076] Specific implementation method three:

[0078] Different from the second specific implementation mode, in the emotion recognition method based on empirical mode decomposition and improved multi-scale fuzzy entropy of this implementation mode, the process of calculating the improved multi-scale fuzzy entropy described in step 3 and using the calculated fuzzy entropy as the feature vector is specifically:

[0079] Fuzzy entropy,Fuzzy entropy is an indicator used to measure the uncertainty in fuzzy sets. Its similarity measurement formula uses an exponential function as a fuzzy function. The continuity of the exponential function makes the fuzzy entropy smooth. In addition, fuzzy entropy introduces the concept of fuzzy sets to measure the similarity of two vectors.

[0080] Assume that fuzzy entropy is defined as follows:

[0081] For a given N-dimensional time series [u(1),u(2),…,u(i+m-1)], define the phase space dimension m (m≤n-2) and reconstruct the phase space:

[0082] X(i)=[u(i),u(i+1),…u(i+m-1)]-u0(i), i=1,2,…N-m+1

[0083] Among them, u0(i) is the mean, expressed as:

[0084]

[0085] Defining vectors and The distance is the Chebyshev distance, that is, the maximum absolute value of the difference between the numerical values ​​of each element:

[0086]

[0087] Introducing fuzzy membership n measure The similarity of , for 1≤j≤Nm,j≠i is:

[0088]

[0089] The fuzzy similarity function is defined as:

[0090]

[0091] The obtained fuzzy entropy estimate is:

[0092] FuzzyEn(m,n,r,N)=lnΦ m (r)-lnΦ m+1 (r)

[0093] And design an improved fuzzy entropy calculation method: Considering the inherent multi-time scale of complex EEG signals, design a multi-scale version to calculate fuzzy entropy, use the time coarse-graining method to divide the intrinsic mode function after empirical mode decomposition into non-overlapping segments of size τ, and average the data of each segment; for a given time series x of N data points and parameter τ, construct the coarse-grained time series y(τ) as follows:

[0094]

[0095] Then, the fuzzy entropy of the time series after time coarse-graining is calculated; and the original Gaussian similarity curve is replaced by an S-type similarity curve, which is expressed as:

[0096] The method of introducing two thresholds α and β (0≤β≤α≤1) divides the membership into three non-overlapping areas, namely complete membership, completely incompatible membership and fuzzy membership; the threshold method introduced is as follows Figure 2 As shown, it is expressed as:

[0097] Specific implementation method four:

[0099] Different from the third specific implementation mode, in the emotion recognition method based on empirical mode decomposition and improved multi-scale fuzzy entropy of this implementation mode, the process of calculating the similarity between feature vectors using the maximum mutual information coefficient in step 5 is specifically:

[0100] The maximum mutual information coefficient (MIC) is used to measure the degree of association between two variables, including linear and nonlinear relationships. It is well known that EEG signals are highly complex, and their intrinsic mode functions include both linear and nonlinear relationships. It is one-sided to measure the amount of information contained in a feature from the perspective of linear correlation or nonlinear correlation, so the maximum mutual information coefficient is introduced. This universality and fairness are just suitable for EEG data. For example, when the sample size is large enough, it can capture a variety of interesting associations. The complex relationship between general variables cannot be modeled by just one function, but requires superposition of functions to express it. When the sample size is large enough, similar coefficients can be given for correlations with similar degrees of single noise of different types. The steps for calculating the maximum mutual information coefficient are as follows:

[0101] First, given i and j, the scatter plot composed of XY is gridded with i rows and j columns, and the maximum mutual information value is calculated:

[0102]

[0103] Afterwards, the maximum mutual information value is normalized:

[0104]

[0105] Afterwards, the maximum value of the mutual information under different i and j scales is selected as the MIC value.

[0106] Example:

[0107] The EEG signals of the DEAP dataset of 32 participants watching 40 movie clips (60 seconds each) were processed to extract the features of the time domain, frequency domain and decomposition domain. After the empirical mode decomposition of the decomposition domain, the improved fuzzy entropy algorithm was used. First, time coarse-graining was used, and the threshold was set and the similarity was automatically calculated. The maximum mutual information coefficient was used to sort the relevance of each eigenvalue, and the optimal eigenvector was selected to achieve feature dimensionality reduction to improve the accuracy of the emotion classification model. Finally, the classification accuracy of more than 80% was obtained in the two dimensions of valence and arousal, respectively, verifying the effectiveness of the present invention for emotion classification. The experimental flow chart is shown below. Figure 3 shown.

[0108] Human emotions play a central role in decision-making, social interaction, and the diagnosis of mental illnesses such as depression. Since machines cannot understand personal feelings, emotions need to be analyzed and calculated. The main purpose of emotional computing is to enable artificial intelligence to have the ability to recognize, analyze, and express, to establish a friendly and healthy human-computer environment, and to improve the intelligence level of computers, so that such interactions are more accessible, usable, and effective. Emotional computing is an important part of the field of human-computer interaction and has become a hot topic in human-computer interaction in recent years.

[0109] Compared with non-physiological signals and the autonomic nervous system, EEG recognition methods are more objective and more accurate. Various discrete classification methods for emotions have been proposed, such as the six basic emotions proposed by Ekman and Friesen, and the emotion tree structure proposed by Parrot. Dimensional scales of emotions have also been proposed, and the mainstream one is the valence-arousal scale proposed by Russelld. Emotion recognition based on EEG signals has become the mainstream direction in the field of emotion recognition. With the in-depth study of emotions by humans and the rapid development of electrode technology in EEG acquisition, EEG signals have been widely used in various aspects. Arousal and valence models such as Figure 4 shown.

[0110] Affective computing is an emerging field, and a lot of research has been done on emotion classification. Modern neuroscience and brain analysis studies have shown that emotions play a key role in determining an individual's stress level and personality development. Watching specific video content or listening to specific types of music can cause specific emotions. Six different features were extracted by using stable EEG patterns to identify the dependent and independent emotional states of participants, and the minimum redundancy maximum correlation parameter was used for feature selection. Three features were calculated using EEG signals from the DEAP dataset, and the model achieved an accuracy of 73.5% in two valence and arousal states. Several entropy-based metrics have also been used in different applications, including approximate entropy and sample entropy. Other studies have shown that the sample entropy values ​​of patients in the depression group are greater than those in the normal group, indicating that the EEG signals in the depression group are more complex. Nonlinear features such as fuzzy entropy and fuzzy fractal dimension (FFD) were also used to analyze the resting EEG signals of 60 subjects with different levels of depression. The results showed that KFD has a good predictive ability for depression levels. An article extracted the wavelet entropy and wavelet energy of EEG as features through discrete wavelet transform, and then used KNN and SVM classifiers to identify emotions. The results showed that the recognition accuracy of KNN was higher than that of SVM in both arousal and valence dimensions.

[0111] In summary, there are many methods for emotion classification based on EEG, but they all have certain limitations, including the number of emotional states considered. Although a limited number of participants may achieve higher accuracy, these methods require more careful consideration. To this end, the present invention uses the DEAP dataset, which comes from 32 volunteers, 40 channels are selected and 40 music videos are watched. EEG and physiological signals are recorded, and each participant also scores the video. The huge amount of data and scientific measurement technology provide strong support for emotion classification.

[0112] During the data set and channel selection process, the DEAP data set was selected. The data is divided into two data packets. The first format is 40×40×8064, which means that each user watched 40 videos, and 40 channels of EEG were selected, with a total of 8064 data samples collected. The second data contains four target labels (valence, arousal, dominance, and liking) for each video. After watching the video, each volunteer scored the video according to the target label, ranging from 1 to 10. The EEG electrodes are placed according to the 10-20 system, distributed at different locations on the scalp, and each location is identified according to different symbols and numbers. It has been shown that the left and right hemispheres and brain lateralization play a prominent role in inducing emotions, while the prefrontal cortex has a significant effect on emotion regulation. It is known that 5 electrodes (P3, FC2, AF3, O1, and FP1) have achieved significant accuracy in emotion recognition. This shows that data from some areas of the brain are more important for emotion classification than other areas. In the model proposed in the present invention, a grid search algorithm is used to select four EEG signal channels with the best performance, and four electrodes FP1, FP2, F3, and C4 are selected. The 32-channel electrode distribution diagram used in the present invention is as follows: Figure 5 As shown;

[0113] EEG signal preprocessing The preprocessing process of the data includes: 1. The data is downsampled to 128Hz. 2. Eog artifacts are removed. 3. A bandpass filter of 4.0-45.0Hz is applied. 4. The data is averaged as a common reference. 5. The data is divided into a 60-second trial and a 3-second pre-experimental baseline. 6. The 24th to 40th seconds of the trial time are selected as experimental data.

[0114] The feature extraction process includes time domain feature extraction, decomposition domain feature extraction and frequency domain feature extraction.

[0115] The time domain feature extraction process is:

[0116] The Hjorth parameter is extracted in the time domain as a feature. The Hjorth parameter includes three feature sets: activity (Ah), mobility (Mh) and complexity (Ch). The Hjorth parameter is calculated as:

[0117] A h =var(s j ) (1)

[0118]

[0119] The activity is defined as the variance of the input signal, var(s j ′) represents the first-order derivative of the variance of the input signal.

[0120] The decomposition domain feature extraction process is:

[0121] (1) Empirical Mode Decomposition: Since EEG signals are nonlinear and non-stationary, at any given time, EEG signals may produce several different oscillation modes, which may lead to large deviations in the measurement based on fuzzy entropy. The intrinsic mode function of empirical mode decomposition contains local feature information of different time scales, and can also reflect the trend of signal superposition oscillation, so EMD is introduced: It provides an effective tool for detecting information from nonlinear and non-stationary processes because it adopts an adaptive calculation method based on the local feature time scale of the data. For a given non-stationary time series x(t), EMD adaptively decomposes the time series into a set of IMFs and a residual, where the IMFs have the same extreme values ​​and number of zero crossings, and are symmetric with respect to the local zero mean. EMD decomposes the time series x(t) into a set of IMFs and residuals, as shown in the equation: In the present invention, the present invention only extracts the first three IMFs and calculates the fuzzy entropy. The principle diagram of empirical mode decomposition is shown in Figure 6 As shown:

[0122]

[0123] (2) Fuzzy entropy. Fuzzy entropy is an indicator used to measure the uncertainty in fuzzy sets. Its similarity measurement formula uses an exponential function as a fuzzy function. The continuity of the exponential function makes the fuzzy entropy smooth. In addition, fuzzy entropy introduces the concept of fuzzy sets to measure the similarity of two vectors. The fuzzy entropy is defined as follows. The present invention extracts fuzzy entropy features from the IMF obtained after empirical mode decomposition. The fuzzy entropy values ​​of each component after calculation are as follows: Figure 7 As shown;

[0124] For a given N-dimensional time series [u(1),u(2),…,u(i+m-1)], define the phase space dimension m (m≤n-2) and reconstruct the phase space:

[0125] X(i)=[u(i),u(i+1),…u(i+m-1)]-u0(i),i=1,2,…N-m+1 (5)

[0126] Where u0(i) is the mean:

[0127]

[0128] Defining vectors and The distance is the Chebyshev distance, that is, the maximum absolute value of the difference between the numerical values ​​of each element:

[0129]

[0130] Introducing fuzzy membership n measure , Similarity. For 1≤j≤Nm,j≠i, we have:

[0131]

[0132] The fuzzy similarity function is defined as:

[0133]

[0134] The obtained fuzzy entropy estimate is:

[0135] FuzzyEn(m,n,r,N)=lnΦ m (r)-lnΦ m+1 (r) (11)

[0136] Improved fuzzy entropy: Considering the inherent multi-time scale of complex EEG signals, the present invention provides a multi-scale version to calculate fuzzy entropy. The temporal coarse-graining method is used to divide the intrinsic mode function after empirical mode decomposition into non-overlapping segments of size τ, and the data of each segment is averaged. For a given time series x of N data points and parameter τ, the coarse-grained time series y(τ) is constructed as follows:

[0137]

[0138] In the present invention, the present invention selects τ=2. Then the present invention calculates the fuzzy entropy of the time series after time coarse-graining. And replace the original similarity curve with an unchanged s-shaped similarity curve. For the similarity function, its specific parameter settings are shown in Table 3.1: Since the distance in the similarity curve is defined as the Chebyshev distance, its specific value is related to the standard deviation of the sequence. The larger the standard deviation of the sequence, the larger the corresponding Chebyshev distance is generally; conversely, the smaller the Chebyshev distance is. For different sequences, the standard deviation is different. If the standard deviation is small, the value of the Chebyshev distance with a large value is judged as an artifact; if the standard of the sequence is large, the value of the Chebyshev distance with a small value is judged as an artifact. In order to formulate an evaluation standard for artifacts, the present invention selects a method of introducing two thresholds of α and β (0≤β≤α≤1), and divides the membership into three non-overlapping areas, corresponding to Figure 8 Positions 1, 2 and 3 in , and the threshold method introduced is also the same Figure 8 As shown;

[0139]

[0140] Table 1 Similarity parameter setting table

[0141] The frequency domain feature extraction process is:

[0142] EEG signal, i.e., electroencephalogram signal, is a non-stationary, nonlinear complex signal, which contains a large amount of physiological and psychological information. Power spectral density (PSD) describes the energy distribution of the signal at different frequencies and is one of the important features of EEG signal analysis. Fourier transform is a method of converting time domain signals into frequency domain signals, which plays an important role in extracting the power spectral density of EEG signals. Therefore, the present invention first performs Fourier transform on the signal and extracts the power spectral density of the signal, and finally solves the power of the Alpha, Beta, and Gamma frequency bands as features. The calculation time of power spectral density is relatively short, but many studies believe that combining PSD features can significantly improve the performance of the classifier.

[0143] Feature selection and classification algorithms

[0144] The process of feature selection algorithm is to select features after feature extraction. A good feature selection method can reduce the redundancy of data, remove irrelevant data, and improve the accuracy of learning model. Fig. 9 As shown, the maximum mutual information coefficient (MIC) is used to measure the degree of association between two variables, including linear and nonlinear relationships. It is well known that EEG signals are highly complex, and their intrinsic mode functions include both linear and nonlinear relationships. It is one-sided to measure the amount of information contained in a feature from the perspective of linear correlation or nonlinear correlation alone, so the present invention introduces the maximum mutual information coefficient. This characteristic of universality and fairness is just suitable for EEG data. For example, when the sample size is large enough, a variety of interesting associations can be captured. The complex relationship between general variables cannot be modeled by just one function, but requires superposition of functions to express; when the sample size is large enough, similar coefficients can be given for correlations with similar degrees of single noise of different types. The steps for calculating the maximum mutual information coefficient are as follows:

[0145] 1. Given i and j, grid the scatter plot composed of XY into i rows and j columns, and find the maximum mutual information value:

[0146]

[0147] 2. Normalize the maximum mutual information value:

[0148]

[0149] 3. Select the maximum value of mutual information under different i and j scales as the MIC value.

[0150] Random Forest Classification Algorithm

[0151] The random forest algorithm is an ensemble learning technique that generates a large number of decision trees during training, generating different classes of patterns for classification tasks. The individual predictions of the decision trees that make up the random forest model provide the basis for the model's predictions. The present invention represents the random forest as For classification tasks, it can be expressed as:

[0152]

[0153] where Y1, Y2, …Y n is the predicted value of the decision tree of each decision tree. The predicted value is the average of the predicted values ​​of each decision tree, which can be expressed as:

[0154]

[0155] Where n is the number of decision trees in the random forest set. By reducing fitting and using the model for classification and average regression, the overall flexibility and accuracy of the model can be improved. The random forest parameter settings are shown in Table 2:

[0156]

[0157] Table 2 Random forest parameter settings

[0158] In the method proposed in the present invention, the present invention uses partial data from all subjects, and selects data with more extreme scores in the label set as experimental data. Data from four electrodes were used in the experiment, and these four electrodes are of great significance for emotion classification. Based on the MIC feature selection and multi-domain feature fusion method, the EEG signal was classified into emotions on the DEAP dataset. The present invention adds a time coarse-graining method before extracting fuzzy entropy and improves the similarity function. Finally, a classification accuracy of more than 80% was obtained in the two dimensions of valence and arousal of the test set. Therefore, it is shown that the feature extraction and feature selection methods mentioned in the present invention are beneficial to the problem of emotion classification.

[0159] The present invention uses a combination of multi-domain features, Hjorth, fuzzy entropy, and frequency band power. However, for emotional computing, detecting more emotions can better understand human psychology through machines. In the future, the present invention intends to use physiological data other than EEG to further improve classification accuracy.

Claims

1. An emotion recognition method based on empirical mode decomposition and improved multi-scale fuzzy entropy, characterized by: The method is implemented by the following steps: Step 1: Select the DEAP dataset and the six channels of the frontal lobe and the two channels of the temporal lobe related to emotion classification as the raw data, and import them into EEGlab for preprocessing. The preprocessing methods include: data downsampling, removal of EOG artifacts, 4.0-45Hz bandpass filtering, data averaging common reference, and event marking; Step 2: Use empirical mode decomposition to decompose the preprocessed data under each stimulus to decompose the IMF of each frequency band; Step 3: Calculate the fuzzy entropy of the IMF of each frequency band using the improved multi-scale fuzzy entropy, and use the calculated fuzzy entropy as the feature vector; Step 4: Use the maximum mutual information coefficient to calculate the similarity between feature vectors, sort the maximum mutual information coefficient values ​​between features in descending order, and find the feature vector with the highest ranking that represents the most classification characteristics as the optimal feature vector; Step 5: Input the optimal feature vector into SVM, input the sample into high-dimensional space, find the optimal classification surface, and achieve sample separation.

2. The emotion recognition method based on empirical mode decomposition and improved multi-scale fuzzy entropy according to claim 1, characterized in that: The process of using empirical mode decomposition to decompose the pre-processed data of each subject under each stimulation segment and decomposing the IMF of each frequency band as described in step 2 is specifically as follows: For a given non-stationary time series x(t), EMD adaptively decomposes the time series into a set of IMFs and a residual, where the IMFs have the same extreme values ​​and number of zero crossings and are symmetric with respect to the local zero mean; EMD decomposes the time series x(t) into a set of IMFs and residuals, as shown in the equation:

3. The emotion recognition method based on empirical mode decomposition and improved multi-scale fuzzy entropy according to claim 1 or 2, characterized in that: The process of calculating the improved multi-scale fuzzy entropy described in step 3 and using the calculated fuzzy entropy as the feature vector is specifically as follows: Fuzzy entropy, fuzzy entropy is an indicator used to measure the uncertainty in fuzzy sets. Its similarity measurement formula uses an exponential function as a fuzzy function. The continuity of the exponential function makes the fuzzy entropy smooth. In addition, fuzzy entropy introduces the concept of fuzzy sets to measure the similarity of two vectors. Assume that fuzzy entropy is defined as follows: For a given N-dimensional time series [u(1),u(2),…,u(i+m-1)], define the phase space dimension m (m≤n-2) and reconstruct the phase space: X(i)=[u(i),u(i+1),…u(i+m-1)]-u0(i), i=1,2,…N-m+1 Among them, u0(i) is the mean, expressed as: Defining vectors and The distance is the Chebyshev distance, that is, the maximum absolute value of the difference between the numerical values ​​of each element: Introducing fuzzy membership n measure The similarity of , for 1≤j≤Nm,j≠i is: The fuzzy similarity function is defined as: The obtained fuzzy entropy estimate is: FuzzyEn(m,n,r,N)=lnΦ m (r)-lnΦ m+1 (r) And design an improved fuzzy entropy calculation method: the time coarse-grained method is used to divide the intrinsic mode function after empirical mode decomposition into non-overlapping segments of size τ, and the data of each segment is averaged; for a given time series x of N data points and parameter τ, the coarse-grained time series y(τ) is constructed as follows: Then, the fuzzy entropy of the time series after time coarse-graining is calculated; and the original Gaussian similarity curve is replaced by an S-type similarity curve, which is expressed as: The method of introducing two thresholds α and β (0≤β≤α≤1) divides the membership into three non-overlapping areas, namely complete membership, completely incompatible membership and fuzzy membership; the introduced threshold method is expressed as:

4. The emotion recognition method based on empirical mode decomposition and improved multi-scale fuzzy entropy according to claim 3, characterized in that: The process of calculating the similarity between feature vectors using the maximum mutual information coefficient described in step 5 is specifically: The maximum mutual information coefficient is used to measure the degree of association between two variables, including linear and nonlinear relationships. The maximum mutual information coefficient calculation steps are as follows: First, given i and j, the scatter plot composed of XY is gridded with i rows and j columns, and the maximum mutual information value is calculated: Afterwards, the maximum mutual information value is normalized: Afterwards, the maximum value of the mutual information under different i and j scales is selected as the MIC value.

Images