An emotion classification and recognition method with an adaptive window for electroencephalogram signals

Abstract

The present invention belongs to the technical field of signal processing, and particularly relates to a method for emotion classification and recognition with an adaptive window for electroencephalogram signals. The specific method is as follows: After preprocessing the collected electroencephalogram data under different emotional states, the generalized orthogonal partial directed coherence method is used to iteratively compare the electroencephalogram signals at different time points and with different lengths, and the key electroencephalogram signals that can best represent emotions are selected; features such as fractal dimension, differential entropy, and power spectral density are extracted according to the selected key electroencephalogram signals; the reliefF algorithm is used to calculate the weights of the extracted features to obtain high-quality features; finally, emotion classification and recognition are performed on the valence and arousal two-dimensional emotion model using the support vector machine algorithm and the K-nearest neighbor algorithm. Through this method, not only can the emotion recognition rate be improved, but also while reducing the data volume, the processing time and calculation cost can be reduced, thereby improving the emotion classification and recognition performance.

Description

Technical Field

[0001] The present invention belongs to the technical field of signal processing, and more particularly relates to a method for emotion classification and recognition with an adaptive window for electroencephalogram (EEG) signals. Background Art

[0002] Among numerous physiological electrical signals, since EEG signals are collected based on brain activities, can directly reflect the activity state of the brain, and have advantages such as convenient collection, high time resolution, and low cost, they are used for emotion recognition research. Currently, research on emotion recognition using EEG signals not only involves artificial intelligence and computer science, but also many interdisciplinary fields such as neuroscience and psychiatry. Studying emotion-related brain cognitive activities is of great significance for people to understand their own emotions, optimize computer-aided functions, develop portable personal health care and monitoring systems, and promote the development of psychological science.

[0003] Emotion EEG data is collected through video induction. Since it is difficult to maintain a relatively stable connection pattern of the functional brain network of the subject in the initial stage of data collection and at the end stage of the experimental task, and during the collection process, the subject may, due to their own reasons, such as drowsiness, fatigue, etc., cause artifact interference, or it is difficult to reflect the human brain emotion processing effect of the subject itself due to excessive calculation time and other aspects of influence. Therefore, if the complete EEG signal is used for emotion classification and recognition, better experimental results cannot be achieved.

[0004] In addition, since EEG signals are high-dimensional data signals, during experiments, they have a large amount of calculation, high cost, and low data signal-to-noise ratio. Therefore, using the complete EEG signal will also make the experimental process more complex. In addition, due to individual differences, different subjects have different response times to different video stimuli. To solve these problems, the present invention proposes a method for emotion classification and recognition with an adaptive window for EEG signals. Summary of the Invention

[0005] In view of the above situation, the present invention proposes a method for emotion classification and recognition with an adaptive window for EEG signals. After preprocessing the collected EEG signals in different emotional states, the key EEG signals that can best represent emotions are selected through the generalized orthogonal partial directed coherence method. On this basis, feature optimization is carried out, and the optimized features are used for emotion classification and recognition, thereby improving the performance of emotion classification and recognition.

[0006] The present invention proposes a method for emotion classification and recognition with an adaptive window for EEG signals, which is characterized by including the following steps:

[0007] Step 1: Collect EEG signals in different emotional states, including but not limited to emotional states such as happiness and sadness;

[0008] Step 2. Preprocessing of EEG data: The original EEG signals contain some artifact interference components, and the artifacts in the signals need to be removed;

[0009] Step 3. Calculating all possible signal combinations in the adaptive window: Use the preprocessed EEG data for data reduction processing in the adaptive window. Denote the minimum window, maximum window, and variation constant as W min 、W max and C respectively. First, set the window size to W min , and find all signal combinations of size W min . Next, increase the window size by the variation constant C. Similarly, find all signal combinations after the size is increased by C. Repeat the previous step until the window size is greater than or equal to W max , and iterate this process to ensure that all possible signal time positions are considered;

[0010] Step 4. Selecting the signal window with the maximum emotional intensity: Calculate the generalized orthogonalized partial directed coherence (gOPDC) value of all signal combinations in the time dimension. Among all window data matrices, select the window data with the highest generalized orthogonalized partial directed coherence value and denote it as W gOPDC ;

[0011] Step 5. Extracting EEG features such as fractal dimension, differential entropy feature, and power spectral density according to the selected window data;

[0012] Step 6. Feature selection using the reliefF algorithm: Use the reliefF algorithm to select an instance feature and find K features corresponding to the same category and K features of different categories. Calculate the weight vectors corresponding to these features, and select the features with the highest quality according to the weight vectors, so that the number of selected features is less than the number of samples;

[0013] Step 7. Using a classifier for emotion classification and recognition: According to the selected features, use the support vector machine (SVM) and K-nearest neighbor (KNN) algorithms to perform emotion classification and recognition on all preprocessed EEG data and the key EEG data after data reduction in the valence and arousal dimensions.

[0014] Among them, the method for calculating the generalized orthogonalized partial directed coherence in Step 4 is based on the multivariate autoregressive model. The multivariate autoregressive model of order p is expressed as:

[0015]

[0016] Among them, m represents the number of channels, X(n)=(x i(n),..., x m (n)) T is a given time series, U(n) = (u i (n),..., u m (n)) T is a white noise vector with a normal distribution, and A r is a prediction coefficient matrix, given by Equation (2):

[0017]

[0018] After establishing the multivariate autoregressive model, the coefficient matrix A of the multivariate autoregressive model is obtained by using the dual extended Kalman filter algorithm r , and perform a Laplace transform on A r to convert it to the frequency domain:

[0019]

[0020] In Equation (3), I is the identity matrix, r is the model order, p is the maximum prediction order of the multivariate autoregressive model, k is the number of explanatory variables in the multivariate autoregressive model, f is the frequency, and the multivariate autoregressive model established for the multi-channel EEG signal in the time domain according to Equation (1) is converted to the frequency domain:

[0021]

[0022] The partial directed coherence value from channel i to channel j is expressed as:

[0023]

[0024] where a j (n, f) is the j-th column of A(n, f), and A ij (n, f) is the ij-th element of A(n, f), is the conjugate transpose vector of a j , P ij (n, f) takes values between 0 and 1, and A ij (n, f) is the coefficient corresponding to the i-th row and j-th column in the coefficient matrix A(n, f),

[0025]

[0026] The value of the generalized orthogonal partial directed coherence is expressed as:

[0027]

[0028] where n is the length of the time series, f is the frequency, λ kk is the eigenvalue of the k-th row and k-th column of the coefficient matrix, ω is a zero-mean white noise vector of the diagonal covariance matrix.

[0029] The technical solution provided by the present invention, compared with the existing technologies, has the beneficial effects that:

[0030] (1) By using an adaptive window to select the part of EEG signals that can better represent emotions, not only can the data be reduced, the calculation amount and cost be decreased, but also the accuracy of emotion classification and recognition can be improved, making the expression of emotions more accurate;

[0031] (2) When this method uses the generalized orthogonal partial directed coherence method to select key EEG signals, it takes into account the relationship between EEG channels, restores the spatial and functional connections of the data itself, and can provide more discriminative emotion information;

[0032] (3) Since the number of data points in the dataset available for EEG emotion classification and recognition is limited, if the number of feature points is significantly higher than the number of data points, it will lead to overfitting of the model. To overcome the overfitting problem, feature selection can reduce the number of feature points required for training the model. Therefore, feature selection is performed through the reliefF algorithm to select a new set of features with the largest amount of emotion information. Description of the Drawings

[0033] The attached drawing is an implementation flowchart of a method for emotion classification and recognition of EEG signals with an adaptive window Detailed Embodiment

[0034] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the following further details the specific embodiments of the present invention in conjunction with the attached drawings.

[0035] Step 1: Collection of EEG signals. Using video induction as a stimulus, collect the EEG data of the subjects' emotional states when watching different videos, including but not limited to emotional states such as happiness and sadness;

[0036] Step 2: Preprocessing of EEG data. Considering that the original EEG signals include some artifact interference components, use the Matlab automatic artifact removal toolbox to remove the artifacts in the signals. First, use blind source separation (BSS) to decompose the original EEG signal X into spatial components, aiming to separate the artifacts caused by brain activities; second, detect the artifact components; finally, reconstruct the EEG data using the non-artifact components. The data is downsampled to obtain a sampling frequency of 128 Hz, reducing the noise-related components in most of the electrode signals;

[0037] Step 3: Calculate all possible signal combinations in the adaptive window. Using all available EEG data is computationally expensive and does not yield a high emotion classification recognition effect. In addition, the emotion-evoking stimuli are lengthy, and during this time, the subjects can experience multiple emotions of different intensities. Therefore, a short time window needs to be selected to extract signals that can better represent emotions;

[0038] Perform data reduction processing on the adaptive window using the preprocessed EEG data. Denote the minimum window, maximum window, and change constant as W min 、W max and C respectively. First, set the window size to W min , and find all signal combinations of size W min . Next, increment the window size by the change constant C. Similarly, find all signal combinations after the size is increased by C. Repeat the previous step until the window size is greater than or equal to W max . Iterating this process ensures that all possible signal time positions are considered;

[0039] Step 4: Select the signal with the maximum emotion intensity. Assume a dataset has S subjects, each subject has M samples, each sample has a duration of t seconds, and the number of sample channels is N. Calculate the generalized orthogonal partial directed coherence value gOPDC between every two channels in the time dimension for all signal combinations of each sample. Add the gOPDC matrices between the selected window data channels. Among all window data matrices, select the window data with the highest generalized orthogonal partial directed coherence value and denote it as W gOPDC ;

[0040] Step 5: Extract EEG features such as fractal dimension, differential entropy feature, and power spectral density according to the selected window data;

[0041] Step 6: Use the reliefF algorithm for feature selection. Use the reliefF algorithm to select an instance feature and find K features corresponding to the same category and K features of different categories. Calculate the weight vectors corresponding to these features. Select the features with the highest quality according to the weight vectors, such that the number of selected features is less than the number of samples. Since the number of data points in the available EEG emotion recognition dataset is limited, and the number of feature points is significantly higher than the number of data points, resulting in overfitting of the model. To overcome the overfitting problem, when performing feature selection, the number of feature points required for training the model can be reduced. Use the reliefF algorithm to select a new set of features with the largest emotion information content. This algorithm has noise resistance and robustness to feature interactions;

[0042] Step 7: Use a classifier for emotion classification and recognition. Based on the selected features, use the support vector machine and the K-nearest neighbor algorithm to perform emotion classification and recognition on all preprocessed EEG data and the key EEG data after data reduction in the valence and arousal dimensions.

[0043] The above-mentioned calculation of the generalized orthogonal partial directed coherence method in Step 4 is based on a multivariate autoregressive model. The multivariate autoregressive model of order p is expressed as:

[0044]

[0045] where m represents the number of channels, X(n) = (x i (n),..., x m (n)) T is the given time series, U(n) = (u i (n),..., u m (n)) T is a white noise vector with a normal distribution, A r is the prediction coefficient matrix, given by Equation (2):

[0046]

[0047] After establishing the multivariate autoregressive model, the coefficient matrix A of the multivariate autoregressive model is obtained by using the dual extended Kalman filter algorithm. r For A r perform a Laplace transform to convert it to the frequency domain:

[0048]

[0049] In Equation (3), I is the identity matrix, r is the model order, p is the maximum prediction order of the multivariate autoregressive model, k is the number of explanatory variables in the multivariate autoregressive model, f is the frequency. For the multivariate autoregressive model established for the multi-channel EEG signal in the time domain according to Equation (1), convert it to the frequency domain:

[0050]

[0051] The partial directed coherence value from channel i to channel j is expressed as:

[0052]

[0053] where a j (n, f) is the j-th column of A(n, f), A ij (n, f) is the ij-th element of A(n, f), is the conjugate transpose vector of a j , and P ij(n, f) takes a value between 0 and 1, A ij (n, f) is the coefficient corresponding to the i-th row and j-th column in the coefficient matrix A(n, f),

[0054]

[0055] The value of the generalized orthogonal partial directional coherence is expressed as:

[0056]

[0057] where n is the length of the time series, f is the frequency, λ kk is the eigenvalue of the k-th row and k-th column of the coefficient matrix, ω is a white noise vector with zero mean of the diagonal covariance matrix.

[0058] Although the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, it is not a limitation on the protection scope of the present invention. Those skilled in the art should understand that based on the technical solutions of the present invention, various modifications or deformations that can be made by those skilled in the art without creative labor are still within the protection scope of the present invention.

Claims

1. An emotion classification and recognition method for electroencephalogram (EEG) signals with an adaptive window, characterized in that, it includes the following steps: Step 1: Collect EEG signals in different emotional states; Step 2: EEG data preprocessing: The original EEG signals may contain some artifact interference components, and the artifacts in the signals need to be removed; Step 3: Calculate all possible signal combinations in the adaptive window: Use the preprocessed EEG data for data reduction processing in the adaptive window. Denote the minimum window, maximum window, and change constant as W min , W max and C respectively. First, set the window size to W min , and find all signal combinations of size W min . Next, increase the window size by the change constant C. Similarly, find all signal combinations after the size is increased by C. Repeat the previous step until the window size is greater than or equal to W max . Iterate this process to ensure that all possible signal time positions are considered; Step 4. Select the signal window with the maximum emotional intensity: Calculate the generalized orthogonalized partial directed coherence (gOPDC) of all signal combinations in the time dimension. Among all window data matrices, select the window data with the highest generalized orthogonalized partial directed coherence value and denote it as W gOPDC ; Step 5: Extract EEG signal features based on the selected window data. The EEG signal features include fractal dimension, differential entropy feature, and power spectral density; Step 6: Use the reliefF algorithm for feature selection: Use the reliefF algorithm to select an instance feature and find K features corresponding to the same category and K features of different categories, calculate the weight vectors corresponding to these features, and select the features with the highest quality according to the weight vectors, so that the number of selected features is less than the number of samples; Step 7: Use a classifier for emotion classification and recognition: According to the selected features, use the support vector machine (SVM) and K-nearest neighbor (KNN) algorithms to perform emotion classification and recognition on all preprocessed EEG data and key EEG data after data reduction in the valence and arousal dimensions.

2. The emotion recognition method for EEG information with an adaptive window according to claim 1, characterized in that, the calculation method of the generalized orthogonal partial directed coherence in Step 4 is based on a multivariate autoregressive model. The multivariate autoregressive model of order p is expressed as: where \(m\) represents the number of channels, \(X(n)=(x i (n),\cdots,x m (n)) T is a given time series, \(U(n)=(u i (n),\cdots,u m (n)) T is a white noise vector with a normal distribution, and \(A r is a prediction coefficient matrix, given by Equation (2): After establishing the multivariate autoregressive model, the coefficient matrix A of the multivariate autoregressive model is obtained by using the dual extended Kalman filtering algorithm r , for A r Perform Laplace transform to convert it to the frequency domain: In Equation (3), I is the identity matrix, r is the model order, p is the maximum prediction order of the multivariate autoregressive model, k is the number of explanatory variables in the multivariate autoregressive model, f is the frequency. Based on the multivariate autoregressive model established for multi-channel EEG signals in the time domain according to Equation (1), it is transformed into the frequency domain: The partial directed coherence value from channel i to channel j is expressed as: where a j (n, f) is the j-th column of A(n, f), and A ij (n, f) is the ij-th element of A(n, f), is the conjugate transpose vector of a j , and P ij (n, f) takes values between 0 and 1, and A ij (n, f) is the coefficient corresponding to the i-th row and j-th column in the coefficient matrix A(n, f). The value of the generalized orthogonal partial directed coherence is expressed as: where n is the length of the time series, f is the frequency, and λ kk is the eigenvalue of the k-th row and k-th column of the coefficient matrix, and ω is a white noise vector with zero mean of the diagonal covariance matrix.

Images