A cross-subject EEG emotion recognition method based on Riemannian manifold

Abstract

The application provides a cross-subject electroencephalogram emotion recognition method based on a Riemannian manifold. Current cross-subject emotion recognition models based on electroencephalogram signals have problems such as poor model generalization, low recognition accuracy and the like, and are limited to analyzing amplitude information of the electroencephalogram signals, and cannot effectively reduce the difference between subjects. The application first extracts four frequency bands θ, α, β and γ of the electroencephalogram signals by using an FIR filter, then calculates a PLV matrix between the electroencephalogram signal channels, and aligns the PLV matrix of a task state to the PLV matrix of a resting state on the Riemannian manifold to reduce the difference between different subjects. Finally, the PLV matrix is mapped to a tangent space for classification and recognition. The application performs a valence and arousal two-classification experiment and a valence-arousal space four-classification experiment on a DEAP data set, and the results show that the method is effective.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of signal processing, and relates to a cross-subject electroencephalogram emotion recognition method based on a Riemannian manifold. BACKGROUND

[0002] Emotion is a complex psychological and physiological state, which plays a complex and basic role in human cognition and behavior. Emotion plays a crucial role in people's daily life and is considered to be the most reliable indicator of human thinking, including behavior and communication ability in human socialization. With the continuous development of artificial intelligence, affective computing has been applied to many fields such as medical treatment, games, human-computer interaction and the like. Since the brain electrical signal is a direct acquisition of the brain's cortical electrical signal, it can reflect the change of human emotional state, therefore, emotion recognition based on electroencephalogram (EEG) is becoming the research focus of affective computing at present. At present, emotion recognition based on electroencephalogram can be divided into single-subject emotion recognition and cross-subject emotion recognition. Single-subject emotion recognition refers to experimental research on a single individual, and the data used in the experiment are collected from the individual, and the model trained is also more suitable for the individual, so better emotion recognition effect can be obtained. Different from single-subject emotion recognition, cross-subject emotion recognition refers to emotion recognition research on multiple subjects, and the experimental data come from multiple subjects, and the model trained is used to test other users who do not participate in the model training. Due to the non-stationary of electroencephalogram and the great difference between individuals, the performance of cross-subject emotion recognition is often poor.

[0003] The physiological mechanism of the brain in different emotional states can effectively improve the generalization ability of cross-subject emotion recognition model. Neuroscience shows that different brain regions are responsible for producing different emotional states, and an emotional state is often produced by the cooperation of multiple brain regions. Therefore, how to capture the commonality between subjects is the key to improving the generalization ability of the model. Traditional analysis methods are often based on linear analysis methods and mainly based on amplitude data analysis. However, the brain is a complex nonlinear system, and the collected EEG signal is also a non-stationary nonlinear signal. On this basis, using linear analysis method to analyze EEG signal will produce certain deviation, and due to the differences between individuals, the amplitude of the collected EEG signal changes greatly. Synchronization is the phenomenon that the interaction between multiple oscillation systems or objects gradually tends to be consistent. In brain neuroscience research, information transmission and processing between different brain regions rely on the synchronous oscillation activity of neurons. Phase synchronization is a special form of synchronization, which refers to the phenomenon that the phase relationship of two signals remains stable over time. Phase synchronization is a nonlinear analysis method and has wide application in brain-computer interface, neuroscience, signal processing and other fields. Using phase analysis can reduce the differences between subjects and the influence of amplitude changes caused by eye movement. In addition, for the correlation matrix between brain regions, existing research often studies it in Euclidean space, ignoring its matrix structure and failing to well represent the similarity between matrices. SUMMARY

[0004] The present application is aimed at the deficiencies of the prior art, in order to improve the generalization performance of cross-subject emotion recognition model, the present application proposes a cross-subject EEG emotion recognition analysis method based on Riemannian manifold.

[0005] The present application first uses FIR filter to filter EEG signal, then uses phase locking value (PLV) to represent the correlation between brain regions in task state, and combines it into phase locking value matrix mapped in Riemannian manifold space; secondly, using the EEG signal of each subject in resting state as reference point, moving the PLV matrix in task state to ensure that the distribution difference between the PLV matrices in task state is formed by different task states; finally, mapping the aligned PLV matrix to tangent space, and identifying the emotional state in tangent space.

[0006] A cross-subject EEG emotion recognition method based on Riemannian manifold, the method comprises the following steps:

[0007] Step 1: Select the EEG signal of each subject related to the emotional state of the specific channel, and then filter the EEG signal using the FIR filter to obtain the theta, alpha, beta, and gamma four frequency bands;

[0008] Step 2: Calculate the phase-locked value of each subject in the four frequency bands of each channel under the task state through Hilbert transform, and map the phase-locked value under each frequency band into the Riemann manifold to form the PLV matrix.

[0009] Step 3: Calculate the PLV matrix of the resting state EEG signal of each subject in the four frequency bands, and use it as the reference matrix to perform Riemann alignment on the PLV matrix in the task state in the Riemann manifold.

[0010] Step 4: Calculate the Riemann mean point of the aligned PLV matrix in each frequency band, and use it as the tangent point to map the PLV matrix into the tangent space to obtain the feature vector of the four frequency bands.

[0011] Step 5: In the tangent space, fuse the feature vectors of the four frequency bands, and use support vector machine for classification and recognition.

[0012] The phase-locked value is a kind of index used to describe the phase synchronization of signals, which reflects the consistency of two signals in phase. PLV pays more attention to the phase information of the signal rather than the amplitude information when judging the phase synchronization, so using the PLV method can separate the phase component and amplitude component of the EEG signal, avoiding the influence of the instantaneous amplitude difference of the EEG caused by eye movement, external environment difference, individual difference, etc. PLV reflects the absolute value of the average phase difference between narrow-band signals, so the range of PLV value is 0 to 1. The closer the PLV value is to 1, the more synchronized the two signals in phase, and the closer the PLV value is to 0, the less synchronized the two signals in phase. After FIR filtering of the EEG signal, the instantaneous phase of the EEG signal is calculated by Hilbert transform. First, the EEG signal x(t) is taken as the real part to construct the complex-valued analytic signal as shown in equation (1):

[0013]

[0014] Where is the analytic signal of the EEG signal, A(t) is the instantaneous amplitude, and φ(t) is the instantaneous phase. The Hilbert transform of φ(t) is used, and the imaginary part is calculated as:

[0015]

[0016] Where PV is the Cauchy principal value. The original signal is phase-shifted by π / 2 through Hilbert transform. Therefore, the instantaneous phase is obtained as:

[0017]

[0018] After obtaining the instantaneous phase of the signal, the PLV between the EEG signals is calculated:

[0019]

[0020] Where Δφ jk is the instantaneous phase difference of the signals x j and x k , and N represents the number of sampling points in the time period.

[0021] The calculated PLV matrix is mapped in the Riemann manifold, and the structural characteristics of the PLV matrix in the non-Euclidean space are fully considered, and the Riemann metric provided on the Riemann manifold is used to more accurately represent the similarity between the PLV matrices.

[0022] As preferred, the selected specific channels in step 1 are O2, P8, T8, FC6, F4, F8, AF4, AF3, F7, F3, FC5, T7, P7, and O1.

[0023] As preferred, the task-state PLV matrix of one subject is Riemann-aligned; specifically:

[0024] The PLV matrix M ref of the resting state of each experiment of the subject is calculated as a reference matrix, and then the PLV matrix P i of the task state obtained in each experiment of each subject is Riemann-aligned according to formula (5):

[0025]

[0026] Where P i ' is the aligned PLV matrix. Since the affine invariant metric (AIRM) on the Riemann manifold has the affine invariance property, that is:

[0027] δ A (W T P1W,W T P2W)=δ A (P1,P2) (6)

[0028] In this way, the PLV matrices of each session can still maintain their internal sample relative distance and geometric structure after alignment. However, the data of different subjects will be moved around the resting state matrix, so that the edge distribution difference between the data can be reduced.

[0029] The beneficial effects of the present application are:

[0030] Firstly, the present application uses a PLV matrix to establish the relationship between brain regions in different emotional states, which utilizes the phase information of the EEG signal, and can reduce the amplitude changes caused by the differences between different subjects.

[0031] Secondly, the constructed PLV matrix is mapped in the Riemannian manifold, better utilizing the matrix structure characteristics, and using the Riemannian metric to more accurately measure the distance between the PLV matrices.

[0032] Thirdly, in the Riemannian manifold, the resting state data is taken as a reference point, and the task state PLV matrix is Riemannian aligned, further reducing the differences between different subjects and improving the model generalization performance. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 for the PLV calculation process between channels;

[0034] Figure 2 for the PLV matrix under the task state;

[0035] Figure 3 for the Riemannian alignment of the PLV matrix on the Riemannian manifold. DETAILED DESCRIPTION

[0036] The embodiments of the present application will be described in detail below in conjunction with the accompanying drawings: The present embodiment uses the DEAP data set on the premise of the technical solution of the present application, and gives a detailed implementation manner and specific operation process:

[0037] The present embodiment includes the following steps:

[0038] Step 1, select the 14-channel EEG data (O2, P8, T8, FC6, F4, F8, AF4, AF3, F7, F3, FC5, T7, P7, O1) of each subject in the DEAP data set, and extract the resting state data and the task state data.

[0039] Step 2, use the FIR filter to extract the theta, alpha, beta, gamma and other frequency bands of the EEG signal, and calculate the PLV between each channel under the resting state and the task state to form the PLV matrix, as shown in Figure 1 Figure 2 .

[0040] Step 3, taking the resting state PLV matrix of each subject as a reference point, align the task state PLV matrix on the Riemannian manifold according to formula (5), as shown in Figure 3 .

[0041] Step 4, calculate the Riemannian mean point of the PLV matrix of different frequency bands after Riemannian alignment, and map the PLV matrix to the tangent space with the point as the tangent point. ​

[0042] Step 5, in the tangent space, the feature vectors of the four frequency bands are fused and classified and recognized using a support vector machine. The present application uses the DEAP dataset to perform a two-classification experiment on valence and arousal of cross-subject emotion recognition, and a four-classification experiment in the Valence-Arousal dimension space. In the experimental study, a leave-one-subject-out method (LOSOCV) is used, that is, the data of one subject is selected each time as the test set, and the data of the remaining subjects is used as the training set, and the process is repeated until the data of each subject is tested as the test set. The average value of the test accuracy from all subjects is used as the final experimental result of the model. The experimental results are shown in Table 1

[0043] Table 1. Classification results of cross-subject emotion recognition of DEAP dataset

[0044]

Claims

1. A cross-subject EEG emotion recognition method based on Riemannian manifold, characterized by: The method comprises the following steps: Step 1: Select the EEG signal of a specific channel related to the emotional state of each subject, and then use FIR filter to filter the EEG signal to obtain four frequency bands: θ, α, β, and γ; Step 2: Calculate the phase-locking values ​​between channels in the four frequency bands for each subject in the task state through Hilbert transform, and map the phase-locking values ​​in each frequency band into a PLV matrix on the Riemannian manifold; Step 3: Calculate the PLV matrix of the resting-state EEG signal in the four frequency bands for each subject, and use it as a reference matrix to perform Riemann alignment on the task-state PLV matrix in the Riemann manifold; Step 4: Calculate the Riemann mean point of the aligned PLV matrix in each frequency band and use it as the tangent point to map the PLV matrix into the tangent space to obtain the eigenvectors of the four frequency bands; Step 5: In the tangent space, the feature vectors of the four frequency bands are fused and classified using a support vector machine.

2. The method for cross-subject EEG emotion recognition based on Riemannian manifold according to claim 1, characterized in that: The specific channels selected in step 1 are O2, P8, T8, FC6, F4, F8, AF4, AF3, F7, F3, FC5, T7, P7, and O1.

3. The cross-subject EEG emotion recognition method based on Riemannian manifold according to claim 1, characterized in that: The phase-locking values ​​between the channels of the four frequency bands for each subject in the task state were calculated by Hilbert transform, specifically: First, the EEG signal x(t) is used as the real part to construct the complex-valued analytical signal as shown in formula (1): in is the analytical signal of the EEG signal, A(t) is the instantaneous amplitude, φ(t) is the instantaneous phase; using the Hilbert transform of φ(t), the imaginary part Calculated as: Where PV is the Cauchy principal value; the original signal is transformed into a π / 2 phase by the Hilbert transform; therefore, the instantaneous phase is: After obtaining the instantaneous phase of the signal, the PLV between EEG signals is calculated: where Δφ jk is the signal x j and x k The instantaneous phase difference is , and N represents the number of sampling points in this time period.

4. The method for cross-subject EEG emotion recognition based on Riemannian manifold according to claim 1, characterized in that: The PLV matrix of the task state is Riemann aligned in the Riemann manifold; Specifically: Calculate the PLV matrix of the subject in the resting state in each experiment as the reference matrix M ref , then for each subject's PLV matrix under each task state obtained in each experiment, perform Riemann alignment according to formula (1): The P i is the PLV matrix of the i-th task state, P i ' is the PLV matrix after the alignment of the i-th task state.

Images