A method for EEG emotion recognition based on robust low-rank subspace self-representation features

Abstract

This invention discloses an EEG emotion recognition method based on robust low-rank subspace self-representation features. Unlike traditional fixed-pattern time-frequency domain features, this method uses a data-driven approach to solve for the self-representation of EEG samples in a low-rank subspace. During the self-representation feature solving process, robust principal component analysis is incorporated to separate noise components in the EEG. Furthermore, to fully utilize the effective information from different EEG frequency bands, a data dimensionality reduction method based on tensor Tucker decomposition is proposed, effectively reducing the complexity of self-representation feature extraction and further improving the accuracy of the extracted features. To verify the effectiveness of this feature, experiments were conducted on the publicly available emotion dataset DEAP, and comparisons were made with several state-of-the-art methods. The results show that the proposed feature is superior in both binary and quadruple classification of emotional EEG valence and arousal.

Description

Technical Field

[0001] This invention belongs to the field of signal processing and relates to an EEG emotion recognition method based on robust low-rank subspace self-representation features. Background Technology

[0002] Emotion is a comprehensive state of feeling, thought, and behavior. It includes not only people's psychological reactions to external stimuli due to personal experiences, but also the physiological reactions associated with these psychological reactions. With the increasing demand for human-computer interaction, the ability of machines to accurately analyze users' emotional states has become crucial to the interactive experience.

[0003] In recent years, emotion recognition research based on electroencephalography (EEG) signals has attracted great interest from many interdisciplinary fields, from psychology to engineering. Compared with emotion recognition methods based on other appearances such as facial expressions and gestures, EEG-based methods are more reliable, have higher accuracy, and offer more objective evaluation.

[0004] However, EEG signals suffer from drawbacks such as temporal asymmetry, instability, low signal-to-noise ratio, and uncertain brain region-specific responses. Therefore, EEG-based emotion recognition remains a challenging task. For traditional machine learning methods, extracting salient features related to emotion changes from EEG signals is key to achieving satisfactory emotion recognition performance. Various features extracted from the time domain, frequency domain, and time-frequency domain have been used in many existing studies. Time-domain features include event-related potentials (ERPs), statistical features, Hjorth features, and higher-order cross features. Frequency-domain feature extraction typically decomposes EEG signals into several frequency bands, including the delta band (1-3Hz), theta band (4-7Hz), alpha band (8-13Hz), beta band (14-30Hz), and gamma band (31-50Hz). Many studies have shown that the gamma band is most helpful for emotion classification, and therefore, features from the gamma band are often selected for classification. Commonly used features include differential entropy (DE), approximate entropy, and sample entropy. Commonly used feature extraction methods for time-frequency features include fast Fourier transform and wavelet transform.

[0005] While existing methods have shown some effectiveness in distinguishing emotional states, a drawback is that researchers must invest significant effort in discovering or designing different emotion-related features from raw EEG signals, and there is still no consensus on which feature is most representative. In pattern recognition and signal processing, data is always assumed to contain a specific structure and can be represented by parametric models, thus enabling intelligent representation and processing. Typical methods, such as principal component analysis (PCA), assume that the data resides in a low-rank subspace; therefore, if a self-representation of the EEG data relative to its low-rank subspace can be found, it will be the best feature representation of the original data. However, because EEG signals have a low signal-to-noise ratio, a large amount of noise is often introduced during acquisition, which inevitably affects the accuracy of feature extraction and classification. To eliminate the influence of noise and improve classification accuracy, some methods based on robust PCA have been proposed. Summary of the Invention

[0006] In order to accurately identify different emotional states of subjects from EEG emotion signals, this invention proposes a robust low-rank subspace self-representation feature extraction method for EEG emotion recognition, and a Tucker decomposition-based data dimensionality reduction method to fully utilize frequency band information while reducing the complexity of solving self-representation features.

[0007] This invention first uses wavelet packet transform to separate EEG signals into various frequency bands, calculates their covariance matrices, and then arranges them along the third dimension to form three-dimensional tensor data. Combined with the proposed Tucker dimensionality reduction method, the data dimensionality of each sample is reduced while keeping the subject dimension unchanged. Finally, the self-representation features of each EEG sample are extracted from the tensor data using the proposed self-representation feature extraction method.

[0008] The beneficial effects of this invention are as follows: This method uses a data-driven approach to solve the self-representation matrix of EEG samples in a low-rank subspace, and incorporates robust principal component analysis to separate noise components in the EEG during the self-representation solution process. Furthermore, to fully utilize the effective information from different EEG frequency bands, a data dimensionality reduction method based on tensor Tucker decomposition is proposed, effectively reducing the complexity of self-representation feature extraction and further improving the accuracy of the extracted features. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating the overall implementation process of the present invention;

[0010] Figure 2 A schematic diagram illustrating tensor data dimensionality reduction based on Tucker decomposition;

[0011] Figure 3 This diagram illustrates the impact of dimensionality reduction on classification accuracy. Detailed Implementation

[0012] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings: These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes:

[0013] This invention mainly includes the following steps:

[0014] Step 1: Preprocess the EEG signal by using a sliding time window to segment the EEG signal.

[0015] Step 2: Use wavelet packet transform to extract the time-frequency domain information of the EEG signal to obtain the corresponding reconstructed signals for the four frequency bands θ, α, β, and γ.

[0016] Step 3: Based on the reconstructed signal obtained in Step 2, calculate the covariance of the samples respectively, and construct 3D tensor data of the same type of samples in terms of subject dimension.

[0017] Step 4: Use a data dimensionality reduction method based on tensor Tucker decomposition to reduce the dimensionality of each sample data while keeping the third dimension of the tensor data unchanged.

[0018] Step 5: Based on the dimensionality-reduced tensor data obtained in Step 4, solve for the robust low-rank subspace self-representation features of each sample, and use support vector machines for classification.

[0019] As a preferred option, step 4 is as follows:

[0020] To fully utilize the useful information from each EEG frequency band for classification, the data from the four extracted bands are concatenated for feature extraction. However, this directly increases the dimensionality of the data used for feature extraction. Furthermore, interference information contained in different frequency bands may also interfere with the extraction of effective features. Therefore, appropriate dimensionality reduction is necessary. Typical dimensionality reduction operations, such as principal component analysis, are two-dimensional operations based on matrices and are not suitable for the extracted three-dimensional data. Tensor decomposition, as a typical operation of tensors, provides a convenient tool for multi-dimensional data dimensionality reduction. Tucker decomposition, also known as higher-order singular value decomposition, has the following expression for third-order data:

[0021]

[0022] in, This represents the original multidimensional data, with a size of N1xN2xN3. Let U be a kernel tensor of size r1xr2xr3 (r≤N), matrix U is N1xr1, matrix V is N2xr2, and matrix W is N3xr3.

[0023] In data compression, Tucker can perform the above operations to extract the low-rank kernel tensor of the original data, representing a large amount of original data with a very small volume. Here, a similar idea to principal component analysis is adopted for third-order tensor data composed of samples, by preserving the kernel tensor... The first n dimensions of N1 and N2 are extracted, while the third dimension, i.e., the sample dimension, remains unchanged. This operation does not affect the number of samples. Considering the high-dimensional relationships among multiple samples, dimensionality reduction is performed on each sample, changing the dimension of the feature data to be extracted from N1xN2xN3 to n1xn2xN3 (e.g., ...). Figure 2 (As shown).

[0024] As a preferred option, step 5 is as follows:

[0025] In many specific problems, signals or data are often represented using matrices, making it easier to understand, model, process, and analyze the data. However, real-world data often faces problems such as inaccuracy, corruption, and noise contamination. Data in practical applications is often low-rank, only becoming low-rank due to noise or other reasons. To restore the low-rank nature of the data, researchers have proposed the RPCA method, which involves dividing the original data matrix D into the sum of two matrices:

[0026] D = A + E (1)

[0027] The ideal data A is well-structured, i.e., low-rank, and the noisy component or outliers E are sparse. It has been shown that A can be efficiently estimated by solving the following optimization problem:

[0028]

[0029] The objective function is the rank of matrix A and the zero norm of the noise matrix E, where λ represents the weight of the noise. The rank function and zero norm in the above equation are both non-convex, making it an NP-hard problem. Often, relaxation is applied before solving. Typically, the kernel norm of a matrix is ​​used to approximate its rank, and the 1-norm is used to approximate its zero norm, thus transforming the problem into a convex optimization problem.

[0030]

[0031] In their paper, Liu et al. proposed a more general rank minimization problem to better handle data mixed with noise. The optimization objective function is as follows:

[0032]

[0033] Where A is the basis for linearly spanning the data space, and Z is the low-rank representation of data X with respect to the basis space A. By obtaining the optimal Z and E, the original data can be recovered through AZ or X - E. Since rank(AZ) < rank(Z), AZ is the low-rank representation of the original data. If A = I, the above formula will degenerate into the form of the original RPCA.

[0034] Equation (5) is a basic method for solving subspace representation at the matrix level. Since EEG data has multiple channels and there is a need to process data from multiple subjects simultaneously, it needs to be extended to the multi-dimensional level. The proposed multi-dimensional tensor-based model is as follows, and the goal is to solve the following optimization problem:

[0035]

[0036] Where the function φ() represents constructing different representations Z (v) into a 3D tensor Then rotate its dimensions to NxVxN, where the superscript v represents extracting the v-th slice. As suggested by Liu et al., the error matrix is the vertical concatenation of the u production matrices of each sample along the columns, i.e., E = [E (1) ; E (2) ; …, E (V) , which can force the columns of the error matrix E (V) in each sample to have a common and consistent magnitude. Therefore, by solving the objective function in Equation (6), the self-representation matrix of each sample on the low-rank subspace after separating the noise part can be obtained.

[0037] The above optimization problem (5) can be solved by using the augmented Lagrangian multiplier method (ALM). To adopt the alternating direction minimization strategy for the problem, the objective function needs to be separable. By introducing the auxiliary tensor variable G, the optimization problem can be transformed into minimizing the following unconstrained problem:

[0038]

[0039] Where the matrix Y v and the tensor represent two Lagrangian multipliers, and μ and are penalty parameters.

[0040] To solve the unconstrained optimization problem in the above equation, an alternating minimization scheme is adopted to update Z (v) 、E (v) 、 The detailed steps are as follows, divided into several small steps:

[0041] Z (v) Subproblem: When E (v) and When fixed, the following subproblems will be solved:

[0042]

[0043] By setting the derivative of the above equation to zero, we can obtain Z. (v) The iterative solution is:

[0044] E (v) Sub-problems:

[0045]

[0046] D is constructed by perpendicularly connecting matrices X(v)-X(v)Z(v)+(1 / μ)Yv along the columns. According to Lemma 4.1 in Liu et al., this subproblem has the following solution:

[0047]

[0048] Where D:,i represents the i-th column of matrix D.

[0049] Subproblem: In order to update the tensor Solve the following subproblems:

[0050]

[0051] In addition, the Lagrange multiplier Y and The following updates are required:

[0052]

[0053] After obtaining the self-representation matrix of each sample in its low-rank subspace, it is converted into a vector by the operation ψ(), that is, the autocovariance of the matrix is ​​calculated and the data of its upper triangular part is extracted as a vector.

[0054] like Figure 1 As shown, this implementation includes the following steps:

[0055] Step 1: Preprocess the emotional EEG signals, remove the 3s baseline signal, and use a sliding time window of 4s length with 2s overlap to segment the EEG signal data in the last 20s to obtain EEG emotional samples of the subject in different emotional states.

[0056] Step 2: Use wavelet packets to extract the four frequency bands θ, α, β, γ of each EEG signal and reconstruct the signal.

[0057] Step 3, process the reconstructed EEG signal X θ ,X α ,X β ,X γAccording to the frequency band combination, a new signal X is formed, the covariance matrix of X is calculated, and tensor data is constructed by connecting X of the same type with the third dimension.

[0058] Step 4: Dimensionality reduction is performed on the obtained tensor data using a Tucker decomposition-based method. According to... Figure 3 The effect of dimensionality reduction on accuracy is shown below. Selecting the optimal dimension n, keeping the third dimension (sample dimension) unchanged, and selecting the first n subspace vectors of the first two dimensions, the tensor data is reconstructed to obtain the dimensionality-reduced tensor data, as shown below. Figure 2 As shown.

[0059] Step 5 involves feeding the dimensionality-reduced tensor data into a robust low-rank subspace self-representation feature extraction method. This method separates the noise component from each EEG signal and calculates the self-representation matrix of each EEG sample in its low-rank subspace. Each self-representation matrix is ​​then extracted as a vector and fed into a support vector machine for classification. To verify the effectiveness of the proposed feature extraction method, classification experiments were conducted on the public dataset DEAP. The classification results are shown in Tables 1, 2, and 3.

[0060] Table 1 shows the accuracy of binary classification using valence scoring as a criterion for 32 subjects in the DEAP dataset.

[0061]

[0062]

[0063] Table 2 shows the accuracy of binary classification using arousal scores as a criterion for 32 subjects in the DEAP dataset.

[0064]

[0065] Table 3 shows the accuracy of binary classification based on valence and arousal for 32 subjects in the DEAP dataset.

[0066]

Claims

1. A brainwave emotion recognition method based on robust low-rank subspace self-representation features, characterized in that, The method includes the following steps: Step 1: Preprocess the EEG signals by using a sliding time window to segment the EEG signals; Step 2: Use wavelet packet transform to extract the time-frequency domain information of the EEG signal to obtain the corresponding reconstructed signals for the four frequency bands θ, α, β, and γ. Step 3: Based on the reconstructed signal obtained in Step 2, calculate the covariance of each sample and construct a third-order tensor data for samples of the same type in terms of sample dimension; Step 4: Use a data dimensionality reduction method based on tensor Tucker decomposition to reduce the dimensionality of each sample data while keeping the third dimension of the tensor data unchanged; Step 5: Based on the dimensionality-reduced tensor data obtained in Step 4, use the robust low-rank subspace self-representation feature extraction method to obtain the feature vector of each sample, and use support vector machine for classification; The specific implementation of the Tucker dimensionality reduction method in step 4 is as follows: in, Where N is the number of EEG acquisition channels, V is the total number of samples, and the subspace matrix A∈R NxN , B∈R NxN ,C∈R VxV ; By keeping the dimensions of the core tensor unchanged so that it remains a complete representation of , the first n-dimensional vectors of the first two subspace matrices are extracted respectively, where n < N; while keeping the third subspace C, i.e., the sample subspace, unchanged, and then performing reconstruction. Thus, considering the high-dimensional connections of multiple samples, the dimension of each sample is reduced, changing the dimension of the feature data to be extracted from NxNxV to nxnxV; The feature vector extraction in step 5 specifically involves solving the following optimization problem: Where c = 1…C represents the category of EEG data, and C is the total category of the analyzed EEG samples; the label v = 1…V c V represents the v-th data point. c This represents the total number of Class C EEG samples; Z is the self-representation matrix of each sample of the same class. (v) The constructed third-order tensor, through The nuclear norm number is subject to low-rank restriction; For each sample noise matrix E of the same class (v) The overall noise matrix formed by vertical arrangement is obtained through ||·|| 2,1 Sparsity constraints are imposed on the norm; S c ∈R NxN Self-representation matrix of the same type The mean, through ||·|| F The constraints make the self-representation matrices of the same type similar; X (v) ∈R NxN For input sample tensors The slice matrix at position v in the third dimension corresponds to the v-th EEG sample data, Z. (v) ∈R NxN and E (v) ∈R NxN These are respectively derived from EEG sample data X (v) The self-representation matrix and noise matrix obtained by decomposition; the function Φ() represents the decomposition of all self-representation matrices Z of class c. (v) Constructed as a third-order tensor The operation; λ1 and λ2 are penalty coefficients; The optimization problem is transformed into an unconstrained problem using the augmented Lagrange multiplier method by introducing auxiliary tensor variables. The optimization problem is transformed into minimizing the following unconstrained problem: Where matrix Y v and tensor Let μ and ρ represent two Lagrange multipliers, where μ and ρ are penalty parameters; To solve the unconstrained optimization problem in the above equation, an alternating minimization scheme is adopted, updating Z sequentially. (v) E (v) , Z (v) Subproblem: When E (v) and When fixed, the following subproblems will be solved: By setting the derivative of the above equation to zero, we obtain Z. (v) The iterative solution is: E (v) Sub-problems: Where D is constructed by vertically connecting the matrices X(v)-X(v)Z(v)+(1 / μ)Yv along the columns, this subproblem has the following solution: Where D:,i represents the i-th column of matrix D; Subproblem: In order to update the tensor Solve the following subproblems: In obtaining the self-representation matrix Z of each EEG sample (v) Then, by calculating each self-representation matrix Z (v) The autocovariance matrix of the matrix is ​​used to extract the upper triangular part of each autocovariance matrix into a vector format as the final feature vector corresponding to each EEG sample.

Images