An emotion recognition method based on common and unique EEG feature mining

Abstract

The present invention discloses an emotion recognition method based on common and unique EEG feature mining, comprising the following steps: step 1: collecting EEG emotion data, step 2: preprocessing the EEG emotion data, step 3: establishing an objective function of an emotion recognition model based on common and unique EEG features, step 4: deriving an expression that needs to be iteratively optimized according to the objective function; step 5: inputting the EEG data preprocessed in step 2 into the objective function, and continuously iterating and optimizing according to the updated expression obtained in step 4 to obtain the emotional state label of the subject. In addition, the present invention can specifically utilize the unique features corresponding to a certain type of emotional state and the common features of all emotional states, so that the model can not only have the performance of better identifying emotional states, but also discover the connection between emotional states and specific features, so that the emotional state can be captured more efficiently.

Description

Technical Field

[0001] The present invention belongs to the technical field of electroencephalogram signal processing, and specifically relates to an emotion recognition method based on mining common and unique electroencephalogram features. Background Art

[0002] Human emotions can be observed by the outside world in two ways: one is through external expressions, such as facial expressions, tone of voice, eye contact, body movements, etc., and the other is through instrument analysis of internal physiological signals, such as EEG signals. These two are also the two major data sources for the current research on emotion recognition. However, external expressions can be re-expressed through human "secondary processing", which is disguised and confusing, and may not be the true emotions that people want to express. This will mislead some emotion recognition models and analyze wrong results. Physiological signals cannot be manipulated by human consciousness and will not lie to the outside world. Relatively speaking, they are more credible. Therefore, choosing physiological signals for emotion recognition research can avoid some factors that cause errors in recognition results to a certain extent.

[0003] However, EEG signals are very sensitive, and the EEG signals of the same emotional state may differ between different subjects or in different time periods of the same subject. On the other hand, in the traditional semi-supervised least squares linear regression model, one part is labeled sample data and the other part is unlabeled sample data. The model is trained to predict the labels of the unlabeled sample data. In the common least squares-based linear regression model, the importance of each dimension of the EEG data in the same time period is measured by a weight matrix, and the importance of each dimension of the sample is continuously corrected during the training process. The predicted labels of unlabeled samples are obtained based on this weight matrix. The present invention will improve the performance of the model for the more difficult recognition task across time periods. Summary of the invention

[0004] The purpose of the present invention is to provide an emotion recognition method based on mining common and unique EEG features.

[0005] The emotion recognition method based on common and unique EEG feature mining is characterized by comprising the following steps:

[0006] Step 1: Collect EEG data of the subjects to obtain EEG emotion data of different emotion types.

[0007] Step 2: Preprocess the EEG emotion data to obtain a labeled data set.

[0008] Step 3: Establish the objective function.

[0009] The 2-1 norm and 1 norm of the weight matrix are respectively increased for the rows and columns of the weight matrix in the least squares linear regression model to obtain an extended least squares semi-supervised linear regression model. At the same time, the adjacency matrix is ​​obtained through the K-nearest neighbor algorithm based on the characteristic information contained in the sample itself, and then the objective function is obtained.

[0010] Step 4: Use the labeled data set to jointly iteratively optimize the objective function, and solve the update expressions of the weight matrix W and the predicted label Ft in the model in turn.

[0011] Step 5: Input the preprocessed EEG data of the subject to be tested into the iteratively optimized objective function to obtain the emotion category of the subject to be tested.

[0012] Preferably, in step 1, by inducing different emotional changes in the subjects and collecting their EEG data, EEG emotion data of different emotion types are obtained.

[0013] Preferably, the preprocessing includes sampling the EEG data at a fixed frequency, and filtering the sampled EEG data through a bandpass filter to remove noise and artifacts.

[0014] Preferably, the preprocessing further includes classifying the EEG data after filtering out noise and artifacts into n frequency bands, respectively calculating the differential entropy in each frequency band, and using the differential entropy as the EEG feature in the sample matrix, where n is the number of divided frequency bands.

[0015] Preferably, the bandpass filter is a 1 Hz-75 Hz bandpass filter.

[0016] Preferably, the EEG data after filtering out noise and artifacts is divided into five frequency bands: Delta (1-4 Hz), Theta (4-8 Hz), Alpha (8-14 Hz), Beta (14-31 Hz) and Gamma (31-50 Hz).

[0017] Preferably, the least squares semi-supervised linear regression model obtained in step 3 is:

[0018]

[0019] in, is a labeled feature matrix; is the characteristic matrix of the object being measured; p represents the dimension of the characteristic matrix, n s and n t There are the number of samples in the training set and the number of samples in the test set respectively; W is the weight matrix; It also consists of two parts, Y s is the label of the feature matrix of the training set samples; F tis the predicted label of the emotion category of the object being tested; n = n s +n t Indicates the total number of samples. F=XW;L2=L * -L; is the adjacency matrix obtained by applying the KNN algorithm to the sample matrix X; the diagonal matrix L * Elements in The 1-norm of the weight matrix W represents the interpretation of the unique features of the sentiment tag; the 2,1-norm of the weight matrix W represents the interpretation of the common features between sentiment tags; λ1, λ2, β are all preset model parameters; tr(·) is the trace operation of the matrix.

[0020] Preferably, in step 4, before optimizing the objective function, the weight matrix and the predicted label are initialized; wherein each element in the predicted label is initialized to c is the total number of categories of sentiment labels, and the weight matrix is ​​initialized as: (X T X+γI) -1 X T Y.

[0021] Preferably, the specific method of joint iteration in step 4 is:

[0022] Fixed the predicted label Ft, updated the weight matrix W, and its objective function is:

[0023]

[0024] Fix the weight matrix W, update the predicted label Ft, and its objective function is:

[0025]

[0026] After solving the above formula, we can obtain the predicted label, the unique features corresponding to a certain type of emotional state, and the common features of all emotional states.

[0027] The present invention has the following beneficial effects:

[0028] 1. The weight matrix W in the linear regression model, each column acts on a category in the label, and each element in each column of the weight matrix can just illustrate the contribution of the features in the sample matrix to the final emotion. For this reason, the present invention adds the 1 norm of the weight matrix W in the model to further consolidate this important significance. Similarly, the emotional state will not only have features that are important to itself, that is, unique features, but also have common features that are meaningful to all emotions, so the present invention also adds the 2,1 norm of the weight matrix W in the model. From the overall sense of EEG data, samples of the same type should be close to each other. In this regard, the present invention uses the K-nearest neighbor algorithm to help the model obtain an adjacency matrix based only on the sample features themselves. Based on this, the present application avoids the traditional approach and conducts targeted research on these unique features. It also uses the common features to have a positive effect on all emotional states, which helps to improve the performance of the model in emotion recognition.

[0029] 2. Through iterative optimization training of the model, the present invention can perform more accurate and effective emotion recognition based on the unique characteristics corresponding to each emotional state in the subject's EEG emotions and the common characteristics of all emotional states, thereby performing more effective subsequent operations on the subject based on the recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is the identification flow chart of the present invention.

[0031] Figure 2 Schematic diagram of feature selection for weight matrices 2-1 norm and 1 norm. DETAILED DESCRIPTION

[0032] The present invention will be further described below in conjunction with the accompanying drawings.

[0033] like Figure 1 As shown, an emotion recognition method based on common and unique EEG feature mining includes the following steps:

[0034] Step 1: Collection of EEG emotion data.

[0035] The subjects are connected to the corresponding brain areas through EEG cap leads, and c types of film clips with obvious emotional tendencies are played at different times to induce emotional changes in the subjects, and the collected EEG data are used as the original emotional data set. In this embodiment, there are four emotion categories, namely sadness, fear, happiness, and calmness. The EEG data collected in one of the time periods is taken as the test data of the subject. The EEG data collected in the remaining time periods are marked with emotional labels to determine the objective function.

[0036] In the above technical solution, by playing c kinds of film clips with obvious emotional tendencies at different times, it is ensured that the acquired EEG data is more comprehensive, avoiding the different EEG data for the same emotion expressed in different time periods, thereby affecting the subsequent recognition, and laying the foundation for accurately identifying the emotional state of the subjects based on the EEG data in the later stage.

[0037] Step 2: Preprocess the EEG emotion data.

[0038] Preprocess the EEG data collected in step 1. Each subject's EEG data corresponds to a sample matrix X. Each sample matrix has a corresponding emotion label vector y, and there are c types of emotions in total. Select two different sample matrices, and make them into data Xs with emotion labels and data Xt without emotion labels respectively.

[0039] Specifically, in this embodiment, the collected EEG data is sampled at a sampling rate of 200 Hz, and then the noise and artifacts are filtered out by a 1 Hz-75 Hz bandpass filter, and the differential entropy (DE) is calculated in 5 frequency bands (Delta (1-4 Hz), Theta (4-8 Hz), Alpha (8-14 Hz), Beta (14-31 Hz) and Gamma (31-50 Hz)) as the sample matrix X, and the calculation formula is as follows:

[0040]

[0041] Among them, σ is the standard deviation of the probability density function; μ is the expectation of the probability density function.

[0042] Through the above technical solution, it can be seen that the differential entropy feature is essentially the logarithmic form of the power spectrum density feature, that is, Therefore, the preprocessing of EEG signals can improve the signal-to-noise ratio, thereby improving the preprocessing effect of the data, reducing interference, and laying the foundation for accurate subsequent recognition.

[0043] Among them, the label vector y is the sentiment label corresponding to the sample matrix X.

[0044] Step 3: Establish the objective function

[0045] By using the identity information contained in the sample itself, the adjacency matrix is ​​obtained using the K-nearest neighbor algorithm. At the same time, the rows and columns of the weight matrix in the least squares linear regression model have different interpretations of the sample labels (see Appendix Figure 2 ), respectively increase the 2-1 norm and 1 norm of the weight matrix W, thereby obtaining an extended least squares semi-supervised linear regression model, and based on this, obtain the objective function.

[0046] Specifically, the expanded least squares semi-supervised linear regression model is:

[0047]

[0048] in, It consists of two parts, is a labeled feature matrix; is the feature matrix of the test data of the object under test (the feature matrix of unlabeled samples); p represents the dimension of the feature matrix, n s and n t are the number of labeled samples and the number of unlabeled samples respectively; W is the weight matrix; It also consists of two parts: Ys is the label of the feature matrix of the training set samples, which is encoded in one-hot format; Ft is the predicted label of the emotion category of the tested object (i.e., the predicted label of the feature matrix of the unlabeled samples); n = n s +n t , represents the total number of samples; c represents the total number of categories of sentiment labels. F=XW; L2=L * -L; is the adjacency matrix obtained by applying the KNN algorithm to the sample matrix X; the diagonal matrix L * Elements in The 1-norm of the weight matrix W represents the interpretation of the unique features of the emotion tag; the 2,1-norm of the weight matrix W represents the interpretation of the common features between emotion tags; λ1, λ2, β are all parameters; tr(·) is the trace operation of the matrix. When using this model, the EEG data of the subject is preprocessed as the feature matrix X t Input the model and obtain the predicted label F according to the model t .

[0049] Step 4: Iteratively optimize the objective function

[0050] According to the objective function described in step 3, the label matrix to be predicted Ft and the weight matrix W in the model are optimized iteratively for multiple times;

[0051] Before optimizing the objective function, the predicted labels and weight matrices need to be initialized. The labels of the unlabeled sample matrix are initialized as The weight matrix is ​​initialized as: (X T X+γI) -1 X T Y. Among them, γ is the preset initialization parameter.

[0052] The specific iteration method is as follows:

[0053] Fix the predicted label matrix Ft and update the objective function of the weight matrix W as follows:

[0054]

[0055] First of all, this is a convex optimization problem, which is solved by the proximal gradient algorithm. Let:

[0056]

[0057] in, is a Hilbert space; f(W) and g(W) are both convex functions, where f(W) is a Lipschitz continuous gradient satisfying A convex function; where L f is a Lipschitz constant; ΔW = W t+1 -W t . Performing a second-order expansion of f(W) yields:

[0058]

[0059] make:

[0060]

[0061] Then the optimization problem of the weight matrix W can be written as follows:

[0062]

[0063] At this time, for the optimization model proposed by the present invention, f(W) and g(W) are specifically:

[0064]

[0065] This can further simplify the optimization expression for W:

[0066]

[0067] And W (t) The value of can be found using the following expression:

[0068] W (t) =W t +b t-1 / b t (W t -W t-1 ) (14)

[0069] Among them, b t satisfy: b0, b1 will be initialized to 1, W t is the result of the tth iteration optimization. In this way, the optimization problem of W becomes solving the following expression:

[0070]

[0071] Among them, S ε [·] is the soft threshold algorithm:

[0072]

[0073] Where, ε=β / L f , and the following operations can be performed based on the initialized W1 and W2:

[0074]

[0075]

[0076] Among them, A ii =1 / 2||w i ||2, w i represents the i-th row of W.

[0077] So L f It can be concluded that:

[0078]

[0079] in, Represents the maximum singular value operation.

[0080] After the above derivation, the iterative optimization process of the weight matrix W under the proximal gradient algorithm can be obtained as follows:

[0081]

[0082] Fix the weight matrix W and update the objective function of the predicted label Ft as follows:

[0083]

[0084] Transforming the above formula into a row-by-row operation on Ft yields:

[0085]

[0086] Where M = X t W, M i is the i-th row vector of M. Then solve the predicted label Ft row by row:

[0087]

[0088] Where v = M i ; x is any unlabeled sample.

[0089] The optimization iteration process of Ft and W is repeated many times, thereby completing the optimization iteration of the entire objective function.

[0090] At the same time, the weight matrix W is processed to obtain the unique and common features of the emotional state.

[0091] In order to obtain the common features of the emotional state, we need to process the obtained weight matrix W as follows:

[0092]

[0093] Among them, θ pi Indicates the importance of each feature dimension of the sample matrix, w pi , w j Represents the row vector of W.

[0094] Since EEG data is collected from different frequency bands, further processing is performed to find the frequency band and lead with the highest weight. The importance of each frequency band ω(i) is calculated as follows:

[0095] w(i)=θ (i-1)×s+1 +θ (i-1)×x+2 +…+θ i×s (twenty four)

[0096] Wherein, b is the number of frequency bands when collecting EEG data, s is the number of leads used when collecting EEG data, and b*s=p.

[0097] No. The importance of each lead ψ(i) is calculated as follows:

[0098] ψ(j)=θ j +θ j+s +…+θ j+(b-1)×s (25)

[0099] Use the weight matrix W to obtain the unique characteristics of a certain emotional state:

[0100]

[0101] w ji represents the element in the i-th column and j-th row, represents the sum of the elements in the ith column. The importance of the ith frequency band and lead is obtained in the same way as in (24) and (25).

[0102] Step 5: Input the EEG data preprocessed in step 2 into the iteratively optimized objective function.

[0103] After iterative optimization of the unlabeled feature matrix (EEG signal) after step 2, the predicted label F is obtained. t That is, the emotional state of the subject at the time of collection. At the same time, the predicted label F tThe serial number corresponding to the maximum value is the corresponding emotion category, thus obtaining the emotion category of the subject when the EEG data is collected.

[0104] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the described embodiments. For those skilled in the art, various changes, modifications, substitutions and variations of these embodiments including components are made without departing from the principles and spirit of the present invention, and still fall within the scope of protection of the present invention.

Claims

1. An emotion recognition method based on mining common and unique EEG features, characterized by: The steps include: Step 1: Collecting EEG data of the subjects to obtain EEG emotion data of different emotion types; Step 2: Preprocess the EEG emotion data to obtain a labeled data set; Step 3: Establish the objective function; The 2-1 norm and 1 norm of the weight matrix are added to the rows and columns of the weight matrix in the least squares linear regression model respectively, and the extended least squares semi-supervised linear regression model is obtained; at the same time, according to the characteristic information contained in the sample itself, the adjacency matrix is ​​obtained through the K-nearest neighbor algorithm; and then the objective function is obtained; The least squares semi-supervised linear regression model is: in, is a labeled feature matrix; is the characteristic matrix of the object being measured; p represents the dimension of the characteristic matrix, n s and n t There are the number of samples in the training set and the number of samples in the test set respectively; W is the weight matrix; It also consists of two parts, Y s is the label of the feature matrix of the training set samples; F t is the predicted label of the emotion category of the tested object; n = n s +n t Indicates the total number of samples; F = XW; L2 = L * -L; is the adjacency matrix obtained by applying the KNN algorithm to the sample matrix X; the diagonal matrix L * Elements in The 1-norm of the weight matrix W represents the interpretation of the unique features of the sentiment tag; the 2,1-norm of the weight matrix W represents the interpretation of the common features between sentiment tags; λ1, λ2, β are all preset model parameters; tr(·) is the trace operation of the matrix; Step 4: Use the labeled data set to jointly iteratively optimize the objective function, and solve the update expressions of the weight matrix W and the predicted label Ft in the model in turn; Step 5: Input the preprocessed EEG data of the subject to be tested into the iteratively optimized objective function to obtain the emotion category of the subject to be tested.

2. The emotion recognition method based on common and unique EEG feature mining according to claim 1 is characterized by: In step 1, by inducing the subjects to have different emotional changes and collecting their EEG data, EEG emotion data of different emotion types are obtained.

3. The emotion recognition method based on common and unique EEG feature mining according to claim 1 is characterized by: The preprocessing includes sampling the EEG data at a fixed frequency, and filtering the sampled EEG data through a bandpass filter to remove noise and artifacts.

4. The emotion recognition method based on common and unique EEG feature mining according to claim 3 is characterized by: The preprocessing also includes classifying the EEG data after filtering out noise and artifacts into n frequency bands, and calculating the differential entropy in each frequency band respectively, and the differential entropy is used as the EEG feature in the sample matrix; n is the number of divided frequency bands.

5. The emotion recognition method based on common and unique EEG feature mining according to claim 3 is characterized by: The bandpass filter is a 1 Hz-75 Hz bandpass filter.

6. The emotion recognition method based on common and unique EEG feature mining according to claim 1, characterized in that: After filtering out noise and artifacts, the EEG data are divided into five frequency bands: Delta, Theta, Alpha, Beta and Gamma.

7. The emotion recognition method based on common and unique EEG feature mining according to claim 1, characterized in that: In step 4, before optimizing the objective function, the weight matrix and the predicted label are initialized; each element in the predicted label is initialized to c is the total number of categories of sentiment labels, and the weight matrix is ​​initialized as: (X T X+γI) - 1 X T Y.

8. The emotion recognition method based on common and unique EEG feature mining according to claim 1, characterized in that: The specific method of joint iteration in step 4 is: Fixed the predicted label Ft, updated the weight matrix W, and its objective function is: Fix the weight matrix W, update the predicted label Ft, and its objective function is: s.t.F t ≥0,F1=1 After solving the above formula, we can obtain the predicted label, the unique features corresponding to a certain type of emotional state, and the common features of all emotional states.

Images