Emotion recognition method, device and equipment based on electroencephalogram signals and medium

Abstract

The invention discloses an emotion recognition method, device and equipment based on electroencephalogram signals and a medium, and relates to the technical field of electroencephalogram signal emotion recognition. The method comprises the steps that resting-state electroencephalogram signals and task-state electroencephalogram signals are preprocessed, and the preprocessed resting-state electroencephalogram signals and the preprocessed task-state electroencephalogram signals are obtained; performing feature extraction on the pre-processed resting-state electroencephalogram signals and the pre-processed task-state electroencephalogram signals to obtain resting-state electroencephalogram features and task-state electroencephalogram features; calculating a task state-resting state differential feature and / or a state transition differential feature between a resting state task and a task state task according to the resting state electroencephalogram feature and the task state electroencephalogram feature; and inputting the task state-resting state difference feature and / or the state transition difference feature after the generalized difference processing into a trained emotion recognition model to obtain an emotion recognition prediction result of the target user, thereby improving the emotion recognition precision.

Description

Technical Field

[0001] The present application relates to the technical field of electroencephalogram (EEG) signal emotion recognition, and in particular to an EEG signal-based emotion recognition method, device, equipment and medium. Background Art

[0002] Mild abnormal emotions such as depression and anxiety are difficult to detect and diagnose. People often regard the initial symptoms as stress, fatigue, or temporary unhappiness in life. Because these symptoms are easily misunderstood, coupled with factors such as time and energy, people often expect to fight through subjective will when they feel uncomfortable or are afraid of discrimination from others, and fail to seek professional help in time, thus delaying diagnosis and subsequent treatment. The diagnosis of abnormal emotions such as depression and anxiety mainly relies on subjective scales and doctor interviews, which have problems such as strong subjectivity and low accuracy. Therefore, a high-precision emotion recognition method is urgently needed. Summary of the invention

[0003] The purpose of this application is to provide an emotion recognition method, device, equipment and medium based on EEG signals, which can improve the accuracy of emotion recognition.

[0004] To achieve the above objectives, this application provides the following solutions: In a first aspect, the present application provides an emotion recognition method based on EEG signals, comprising the following steps.

[0005] Obtain the target user's resting-state EEG signals and task-state EEG signals.

[0006] The resting-state EEG signal and the task-state EEG signal are preprocessed respectively to obtain a preprocessed resting-state EEG signal and a preprocessed task-state EEG signal.

[0007] Feature extraction is performed on the preprocessed resting-state EEG signals and the preprocessed task-state EEG signals respectively to obtain resting-state EEG features and task-state EEG features; task-state-resting-state differential features and / or state transition differential features between resting-state tasks and task-state tasks are calculated based on the resting-state EEG features and task-state EEG features.

[0008] based on , perform generalized difference processing on the target features to obtain generalized difference processing features; represents the first system model, is the generalized difference processing feature, Indicates different time periods. is the second system model, is the target feature, which includes the task state-resting state differential feature and / or state transition differential feature between the resting state task and the task state task. For noise.

[0009] The generalized difference processing features are input into the trained emotion recognition model to obtain the emotion recognition prediction result of the target user.

[0010] In a second aspect, the present application provides an emotion recognition device based on EEG signals, comprising the following modules.

[0011] The EEG signal acquisition module is used to obtain the target user's resting state EEG signals and task state EEG signals.

[0012] The preprocessing module is used to preprocess the resting-state EEG signal and the task-state EEG signal respectively to obtain a preprocessed resting-state EEG signal and a preprocessed task-state EEG signal.

[0013] The feature calculation module is used to extract features from the preprocessed resting-state EEG signals and the preprocessed task-state EEG signals, respectively, to obtain resting-state EEG features and task-state EEG features; and to calculate the task-state-resting-state differential features and / or state transition differential features between the resting-state task and the task-state task based on the resting-state EEG features and the task-state EEG features.

[0014] Generalized difference processing module for , perform generalized difference processing on the target features to obtain generalized difference processing features; represents the first system model, is the generalized difference processing feature, Indicates different time periods. is the second system model, is the target feature, which includes the task state-resting state differential feature and / or state transition differential feature between the resting state task and the task state task. For noise.

[0015] The emotion recognition module is used to input the generalized difference processing features into the trained emotion recognition model to obtain the emotion recognition prediction result of the target user.

[0016] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned EEG signal-based emotion recognition method.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned method for emotion recognition based on EEG signals.

[0018] According to the specific embodiments provided by the present application, the present application discloses the following technical effects: the present application provides an emotion recognition method, apparatus, device and medium based on EEG signals. After obtaining the preprocessed resting-state EEG signals and the preprocessed task-state EEG signals, feature extraction is performed on the preprocessed resting-state EEG signals and the preprocessed task-state EEG signals respectively to obtain resting-state EEG features and task-state EEG features. Task-state-resting-state differential features and / or state transition differential features between resting-state tasks and task-state tasks are calculated based on the resting-state EEG features and task-state EEG features. At least one of the task-resting-state differential features and the state transition differential features is generalized differentially processed. The generalized differentially processed features are used as inputs to an emotion recognition model for emotion recognition. A machine learning model is used to learn the rules between the task-resting-state differential features and / or state transition differential features after generalized differential processing and the user's clear recognition results. The trained emotion recognition model is used for emotion recognition, thereby avoiding human subjective influence and being more accurate. Among them, EEG characteristics are non-stationary and nonlinear time series data. The generalized difference processing method is used to process the task state-resting state differential features and / or state transition differential features, which can make them more adaptable and predictive, thereby further improving the prediction accuracy of emotion recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 This is a diagram of the application environment of an emotion recognition method based on EEG signals in one embodiment of the present application.

[0021] Figure 2 A flowchart of an emotion recognition method based on EEG signals is provided in accordance with an embodiment of the present application.

[0022] Figure 3 A schematic diagram of the specific process of an emotion recognition method based on EEG signals provided in one embodiment of the present application.

[0023] Figure 4 A schematic diagram of the functional design of a hardware device provided in one embodiment of the present application.

[0024] Figure 5 A schematic diagram of a preprocessing process provided in an embodiment of the present application.

[0025] Figure 6A schematic diagram of the functional modules of an emotion recognition device based on EEG signals provided in one embodiment of the present application.

[0026] Figure 7 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0029] The emotion recognition method based on EEG signals provided in the embodiments of the present application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 may send the resting-state EEG signal and the task-state EEG signal of the target user to the server 104. After receiving the resting-state EEG signal and the task-state EEG signal of the target user, the server 104 preprocesses the resting-state EEG signal and the task-state EEG signal respectively to obtain the preprocessed resting-state EEG signal and the preprocessed task-state EEG signal, extracts features from the preprocessed resting-state EEG signal and the preprocessed task-state EEG signal respectively to obtain resting-state EEG features and task-state EEG features, calculates the task-resting-state differential features and the state transition differential features between the resting-state task and the task-state task according to the resting-state EEG features and the task-state EEG features, inputs the task-resting-state differential features and / or the state transition differential features after the generalized differential processing into the trained emotion recognition model, and obtains the emotion recognition prediction result of the target user. The server 104 may feed back the obtained emotion recognition prediction result for the target user to the terminal 102. In addition, in some embodiments, the emotion recognition method based on EEG signals can also be implemented independently by the server 104 or the terminal 102. For example, the terminal 102 can directly perform emotion recognition on the resting-state EEG signals and the task-state EEG signals, or the server 104 can obtain the resting-state EEG signals and the task-state EEG signals from the data storage system and perform emotion recognition on the resting-state EEG signals and the task-state EEG signals.

[0030] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0031] In an exemplary embodiment, Figure 2 As shown, a method for emotion recognition based on EEG signals is provided. The method is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used for explanation, and the steps include the following steps 201 to 205.

[0032] Step 201, obtaining the resting state EEG signal and task state EEG signal of the target user.

[0033] Step 202 , preprocessing the resting-state EEG signal and the task-state EEG signal respectively to obtain a preprocessed resting-state EEG signal and a preprocessed task-state EEG signal.

[0034] Step 203, respectively extracting features from the preprocessed resting-state EEG signal and the preprocessed task-state EEG signal to obtain resting-state EEG features and task-state EEG features; calculating task-state-resting-state differential features and / or state transition differential features between resting-state tasks and task-state tasks based on the resting-state EEG features and task-state EEG features.

[0035] Step 204, based on , perform generalized difference processing on the target features to obtain generalized difference processing features; represents the first system model, is the generalized difference processing feature, Indicates different time periods. is the second system model, is the target feature, which includes the task state-resting state differential feature and / or state transition differential feature between the resting state task and the task state task. For noise.

[0036] Step 205, input the generalized differential processing features into the trained emotion recognition model to obtain the emotion recognition prediction results of the target user. In this embodiment, the emotion recognition model can include two meanings: 1. Emotion judgment, such as normal emotions and abnormal emotions; 2. Quantitative evaluation of severity, the severity of abnormal emotions. The emotion recognition prediction results can include the user's emotion judgment results and the severity (degree of possibility) of abnormal emotions. The trained emotion recognition model is a model trained with the generalized differential processing features of sample users as input and the real emotion recognition results of sample users as labels.

[0037] Implement the above steps 201 to 205, use the task state-resting state differential features and the state transition differential features as the input of the emotion recognition model for emotion recognition, and improve the accuracy of emotion recognition. Use a machine learning model to learn the law between the task state-resting state differential features and the state transition differential features and the user's clear recognition results, and use a trained emotion recognition model for emotion recognition, which avoids human subjective influence and is more accurate. In view of the problems of small number of leads, no tasks, low accuracy, etc. in related technologies, the present application provides a lightweight brain emotion detection device and method to achieve accurate identification of abnormal emotions. The technical solution of the present application has technical advantages such as high number of leads, resting state combined with task state emotion induction design, and differential EEG features.

[0038] The above step 201 may include the following steps 301 to 302.

[0039] Step 301: Acquire the target user's resting state original signal and task state original signal; the resting state original signal and the task state original signal are EEG signals collected when the target user performs a resting state task and a task state task respectively. The resting state is usually the state when the eyes are open and there is no task, and the task state is usually the state when the subject receives external stimulation; the external stimulation can be pictures, music, text, video, animation, etc.; the task state can be multiple stimulation tasks connected together in succession, and the time length remains consistent.

[0040] Step 302: Amplify and filter the resting state original signal and the task state original signal respectively to obtain a resting state EEG signal and a task state EEG signal.

[0041] The signal acquisition of the emotion recognition method based on EEG signals provided in this application is implemented based on hardware devices. The functional design of the hardware devices is as follows: Figure 4 As shown, the hardware device includes eight-lead silver electrodes, an analog front end, a microcontroller, a Bluetooth module, a computer terminal, a storage module and a lithium battery.

[0042] The physical design of the signal acquisition module is mainly divided into two parts. The first part is the data host hardware part: the analog front end part is implemented by the ADS1299-8 chip, which amplifies and filters the original signals in the resting state and the original signals in the task state; the microcontroller part is implemented by the STM32F103RCT6 chip; the communication part is completed by the HC42 Bluetooth module, and the data processed by the analog front end is transmitted to the computer, and the computer-side supporting software is used to display the data processed by the analog front end; the power supply part is completed by a lithium battery with a battery voltage of 3.7V, which provides power for the analog front end, microcontroller, and Bluetooth module; the storage module part is completed by the built-in 128M memory. The second part is the signal acquisition electrode part: according to the 10-20 system, 8 silver dry electrodes are placed as active electrodes, and an earlobe ground electrode is set as a reference electrode. The acquisition equipment fixes the EEG activity electrode to the non-hair-covered area of ​​the forehead through elastic cloth, or fixes it to the forehead through a hard shell combined with a rotatable button. The data host is fixed on the elastic cloth or hard shell.

[0043] Software design of multi-channel EEG data acquisition system: A supporting software is designed based on Python and Java languages ​​to analyze and visualize the collected EEG data and generate analysis results. The main functional modules are as follows: 1. Waveform display: This function module can visualize the EEG signal data of the specified channel and the corresponding recording switch in real time, and can adjust the filter range and filter order. It also has fast Fourier transform, offline data playback, data forwarding, and general EEG signal data acquisition functions. 2. Data acquisition: This function module collects EEG data under the induced state, induces emotions through emotional stimulation animations, and collects scales synchronously to obtain EEG signals and scale data. 3. Result presentation: This function module identifies abnormal emotional types of subjects (i.e. users), such as anxiety and depression, based on built-in feature extraction algorithms and artificial intelligence algorithms. 4. User management: This function module manages user information. 5. Data management: Manage the collected EEG signal data. 6. Connection settings: In this function module, set an effective connection between the acquisition device and the computer.

[0044] Signal acquisition paradigm design: In the waveform display module, users only need to wear the acquisition device and ensure that the acquisition device is connected to the computer normally, and then the software can be used to collect online EEG data and save EEG signal data. In the data acquisition module, the brain state recognition of depression and anxiety can be performed. The EEG signal data acquisition is designed in two steps: the first step is to set a 30-second resting task, and the user must concentrate on staring at a crosshair pattern in the center of the computer monitor. The second step: based on the Self-Rating Depression Scale (SDS) and the Self-Rating Anxiety Scale (SAS), 40 emotion-induced dynamic graphics questionnaire questions are designed under the task state. The answer time for each question must be no less than 5 seconds, and each question is only allowed to submit an answer once, and the options cannot be modified after submission.

[0045] Resting state EEG signals and task state EEG signals include EEG signals of multiple rhythmic dimensions, which may include Rhythm, Rhythm, Rhythm, Rhythm and Rhythm. Based on the above acquisition process, the target user's resting EEG signals with a time length of 30 seconds in each rhythm dimension and the task-state EEG signals in each rhythm dimension corresponding to 40 task-state tasks can be obtained.

[0046] In another exemplary embodiment of the present application, Figure 5 As shown, the preprocessing of the resting state EEG signal and the task state EEG signal obtained in step 201 includes downsampling, bandpass filtering, bad segment removal, outlier removal, baseline drift removal and automatic segmentation in sequence, in preparation for subsequent feature extraction and selection.

[0047] In the above step 203, feature extraction is performed on the preprocessed resting-state EEG signal and the preprocessed task-state EEG signal to obtain resting-state EEG features and task-state EEG features, which specifically includes the following steps 401 to 403.

[0048] Step 401: for each rhythm dimension of the preprocessed resting-state EEG signal, calculate the resting-state EEG features of multiple feature index dimensions under the rhythm dimension according to the preprocessed resting-state EEG signal of the rhythm dimension.

[0049] Step 402: For each of the resting-state EEG features of each of the characteristic indicator dimensions of the rhythm dimension, segment the resting-state EEG features of the characteristic indicator dimension of the rhythm dimension according to the time length to obtain a plurality of resting-state segmented features; calculate the mean of all the resting-state segmented features to obtain the resting-state EEG features of the characteristic indicator dimension of the rhythm dimension.

[0050] Step 403: The target user performs multiple task-state tasks to obtain task-state EEG signals corresponding to each task-state task; for the preprocessed task-state EEG signals of each rhythmic dimension of each task-state task, calculate the task-state EEG features of multiple feature indicator dimensions under the rhythmic dimension of the task-state task according to the preprocessed task-state EEG signals of the rhythmic dimension; use the task-state EEG features of each feature indicator dimension of each rhythmic dimension of each task-state task as the task-state EEG features of each feature indicator dimension of each rhythmic dimension of each task-state task.

[0051] The phase lag index (PLI), power spectrum or entropy in the EEG features are selected as identification features, and multi-dimensional EEG feature data is obtained through calculation. The calculation formula of the phase lag index is shown in formula (1).

[0052] (1); In the formula, Indicates Column and The phase lag index of the column EEG signal, the EEG signals are all in matrix form, the columns represent the lead signals between two electrodes, and the present application is provided with eight electrodes, so the matrix has 7 columns; Indicates Column and EEG time series signal The instantaneous phase difference at time, is a sign function, Indicates taking the average, Indicates taking the absolute value. The value of PLI is between 0 and 1. When PLI is 0, it means that there is no phase synchronization effect between the two EEG signals. When PLI is 1, it means that the two EEG signals are phase-locked, and finally the EEG signals are converted into recognizable feature vectors. Synchronously extract the differential EEG features between the resting state and the task state, and between the task state and the task state.

[0053] This application proposes a three-in-one multi-paradigm fusion EEG differential feature extraction technology system of "dynamic baseline correction-cross-paradigm coupling-explainability mining". Based on the EEG data of two paradigms, namely resting state (eyes open baseline) and task state (multi-segment emotion-induced animations), it constructs resting state-task state and task state-task state fusion EEG differential features, which can not only realize the efficient extraction of EEG features, but also improve the interpretability of the features.

[0054] (1) Zero-meaning to eliminate individual baseline drift: In neuroscience, EEG signals have two paradigms: resting state and task state. 1) Resting state features: 30 seconds of resting state, divided into 6 segments of resting state segment features, each segment is 5s, and then averaged to obtain a set of resting state EEG features. 2) Task state features: The EEG features corresponding to each task, the features of all tasks are tiled and spliced. The target user performs multiple task state tasks to obtain the task state EEG signal corresponding to each task state task; for each task state task, the preprocessed task state EEG signal of each rhythm dimension is used to calculate the task state feature values ​​of multiple feature indicator dimensions under the rhythm dimension of the task state task according to the preprocessed task state EEG signal of the rhythm dimension. 3) Task state-resting state differential features: The task state EEG feature corresponding to each task minus the resting state EEG feature.

[0055] The resting-state EEG characteristics can be recorded as ,in is the number of electrodes, Each element of Represents the resting brain area and The characteristic value of the task state EEG feature can be recorded as ,in Indicates Task state, element Indicates task status Brain Region and The eigenvalue of . Contains individual intrinsic neural activity baseline (Right now ,in is zero mean fluctuation). Task state Task-induced changes With baseline The superposition of Direct use or When classifying, the baseline and Masking task-induced specific changes , through the task state-resting state zero mean, the task state-resting state differential feature is expressed as shown in the following formula (2).

[0056] (2).

[0057] If we assume that the individual baseline is consistent across tasks and resting states (i.e. , Here, the resting state must be the eyes-open resting state), then the zero-mean-normalized task state-resting state differential feature is expressed as the following formula (3).

[0058] (3).

[0059] The above proof process shows that zero averaging can eliminate individual baseline drift, retain task-induced relative changes, and enhance the specific representation of emotions by features.

[0060] For different rhythm dimensions, the task state-resting state differential features of all rhythm dimensions can be flattened and connected to obtain the task state-resting state differential features that are finally input into the trained emotion recognition model.

[0061] (2) Differential method suppresses common mode noise and low-frequency interference: The difference of adjacent task data can eliminate non-stationary trends. In the continuous task state, environmental noise or physiological artifacts (such as eye movement, head movement, breathing) have low-frequency common mode characteristics. Define the continuous task state difference, that is, the state transition difference feature It can be expressed as the following formula (4).

[0062] (4).

[0063] Expand equation (4) into the following equation (5).

[0064] (5).

[0065] in, For the task The observed noise due to common mode noise , the post-differentiation noise term The power of It can reflect dynamic neural response patterns, and patients show abnormalities in such dynamic properties (such as reduced flexibility), thereby enhancing classification discriminability.

[0066] For different rhythm dimensions, the state transition differential features of all rhythm dimensions can be flattened and connected to obtain the state transition differential features that are finally input into the trained emotion recognition model.

[0067] The state transition differential characteristics can be calculated by two methods.

[0068] Method 1: Based on the task-resting state features, the difference between the features of the latter task and the previous task is processed (i.e. ).

[0069] Method 2: The task-state EEG features of the kth to nth task-state tasks are flattened and then merged into , and then amortize the PLI features of the k+1~n+1th task state tasks and merge them into , ,Right now , and finally amortize A to obtain the state transition differential feature.

[0070] (3) Mathematical formalization: Generalized difference model of EEG features: Assume that the EEG features obey the system equation shown in equation (6).

[0071] (6).

[0072] in, Indicates different time periods. is the unit shift operator (i.e. ), are model parameters, is the polynomial order. is the first system model, which characterizes the autoregressive characteristics of EEG features, that is, the eigenvalue of the current time period How to rely on its historical value , In both resting and task states, features between brain regions may be time-dependent (e.g., inertia or persistence of neural activity). The coefficient of The strength of this dependency is quantified. For example, When it is large, it indicates that the state at the previous moment has a significant impact on the current state, which may correspond to the stability of the neural network. are model parameters, is the polynomial order. The second system model characterizes the dynamic driving effect of task input on EEG features, that is, external task stimulation How to influence the eigenvalues ​​by different time delays . Task stimulation may trigger transient responses in brain regions ( items) or delayed responses ( Item). For example, This indicates that task input has an immediate modulatory effect on EEG features, which may correspond to the rapid allocation of attentional resources; This indicates that there is a first-order delay in the modulation, which may reflect the conduction time of the neural signal. is the EEG characteristic observation value, Input stimulus for the task, For noise. Use directly This will introduce a baseline bias . Through the difference operator Acting on both sides of the equation, as shown in equation (7).

[0073] (7).

[0074] at this time and The mean has been removed, and is zero-mean white noise. In the experiment, the task input To design task stimuli, the difference can be regarded as pulse excitation, and The transient neural response induced by the extracted task is more sensitive to abnormal EEG characteristics related to depression. The above model can correspond to different tasks by adjusting the order of the system, so that the output differential EEG features have specific neurobiological meanings, which improves the interpretability of the method. The model can be applied through the following two ideas, that is, the generalized differential processing features of step 204 can be obtained through any of the following two ideas.

[0075] Solution 1: Assumption and are all zero-order, that is The target feature can be regarded as the EEG feature observation value , as the input of the subsequent emotion recognition model.

[0076] Solution 2: If and Not zero order, the task-resting state differential features or state transfer differential features between the resting state task and the task-state task Treat it as an external task stimulus and obtain the features that optimize the classification task performance in a data-driven way Considered as , and then solve it by least squares fitting and Parameters of EEG features are obtained by using the generalized difference model The mathematical expression of is used as the input of the subsequent emotion recognition model. The features when the classification task performance is optimal Through feature selection methods.

[0077] In practical applications, the values ​​obtained in (1) and (2) above are It can be used alone or in combination. Any of the task-resting state differential features and state transition differential features between the resting-state task and the task-state task can be processed by generalized differential processing and used as the input of the trained emotion recognition model. The task-resting state differential features and state transition differential features can also be combined and the combined features can be processed by generalized differential processing and used as the input of the trained emotion recognition model. The feature with the best performance in the classification task can be selected by feature selection method. As input to the emotion recognition model.

[0078] In the above step 203, task state-resting state differential features and state transition differential features between resting state tasks and task state tasks are calculated according to resting state EEG features and task state EEG features, which specifically includes the following steps 501 to 503.

[0079] Step 501: For each rhythm dimension and each characteristic indicator dimension of each task-state task, subtract the task-state EEG characteristics and the resting-state EEG characteristics of the rhythm dimension and the characteristic indicator dimension of the task-state task to obtain the task-state-resting-state differential characteristics of the rhythm dimension and the characteristic indicator dimension of the task-state task.

[0080] Step 502: Determine the state transition differential features in each of the characteristic indicator dimensions of each of the rhythm dimensions.

[0081] Step 503: Perform feature selection on the task state-resting state differential features of the characteristic indicator dimension of the rhythm dimension of all the task state tasks and all the state transition differential features to obtain the task state-resting state differential features and state transition differential features between the resting state tasks and the task state tasks.

[0082] The above step 502 can be implemented in any of the following two ways.

[0083] (1) The first method: perform differential operation on the task state-resting state differential features of each rhythm dimension and each characteristic indicator dimension of two adjacent task state tasks to obtain the state transition differential features under each rhythm dimension and each characteristic indicator dimension.

[0084] (2) The second method includes the following steps: Select M consecutive task-state tasks from all task-state tasks and record them as the first task set; M is less than Y, and Y is the total number of task-state tasks; For each of the rhythm dimensions and each of the characteristic indicator dimensions, amortize and merge the task-state EEG features of the rhythm dimension and the characteristic indicator dimension of the M task-state tasks in the first task set to obtain a first merged feature of the rhythm dimension and the characteristic indicator dimension; The next task state task of the first task state task of the selected M task state tasks is taken as the first task state task, and M consecutive task state tasks are selected from all task state tasks again, which are recorded as the second task set; For each of the rhythm dimensions and each of the characteristic indicator dimensions, amortize and merge the task-state EEG features of the rhythm dimension and the characteristic indicator dimension of all task-state tasks in the second task set to obtain a second merged feature of the rhythm dimension and the characteristic indicator dimension; Calculate the correlation between the first task set and the second task set according to the first combined feature and the second combined feature of the feature indicator dimension of the rhythm dimension; The correlation between the first task set and the second task set is amortized to obtain the state transition differential features under each of the rhythm dimensions and each of the feature indicator dimensions.

[0085] Multidimensional EEG features often contain many redundant and irrelevant features, including more redundant features, which will affect the recognition speed and accuracy. Therefore, it is necessary to adopt feature selection methods to delete unimportant features and retain features that contribute more to the model. The recursive feature elimination (RFE) method can be used for feature selection.

[0086] The emotion recognition module in the present application can be a SVM model, a RF model, a KNN model, an XGBoost model, a LightGBM model or an MLP model, etc.

[0087] The trained emotion recognition model is a model trained with the task-resting state differential features and state transition differential features of sample users as input and the real emotion recognition results of sample users as labels. After obtaining the trained emotion recognition model, the task-resting state differential features and state transition differential features between the target user's resting state task and task-state task are input into the trained emotion recognition model to obtain the target user's emotion recognition prediction results. After algorithm recognition processing, a probability value between 0 and 1 will be output, which is positively correlated with the risk of abnormal emotions, that is, the closer the value is to 1, the greater the possibility of illness.

[0088] Based on the SVM model in the machine learning algorithm, the feature data selected in (3) is input into the SVM model to judge and identify the input EEG signal.

[0089] like Figure 3 As shown, the emotion recognition method based on EEG signals provided in this application includes the following steps 1 to 5.

[0090] Step 1: EEG signal acquisition. Executing body: brain state detection equipment and multi-channel EEG data acquisition system software. Description: The brain state detection equipment collects data in normal state through the waveform display function module, and collects data in resting state and task state for depression and anxiety identification through the data acquisition function module. The collected signals are amplified and filtered by analog front-end chips to obtain high-quality original EEG signals, providing a data basis for subsequent processing. The wearable structure design is used to improve the portability and comfort of the wearing process. In the diagnosis and identification process of depression and anxiety, EEG data acquisition stages in resting state and task state are set up to obtain brain state data at different stages.

[0091] Step 2: Signal preprocessing. Execution body: microcontroller and preprocessing algorithm. Description: Downsample, bandpass filter, remove bad segments, remove outliers, remove baseline drift, and automatically segment the data after simple processing, amplification and filtering by the analog front end to further improve the signal quality and prepare for subsequent feature extraction.

[0092] Step 3: Feature extraction and selection. Execution body: Feature extraction and selection algorithm. Description: Phase Lag Index (PLI), power spectrum or entropy in EEG features are used as identification features. PLI features are used to quantify the asymmetry of phase lag or lead between two signals while avoiding volume effects. They are used to study phase synchronization between different brain regions. This feature is calculated and extracted from the preprocessed EEG signal using mathematical formula (1).

[0093] Multidimensional EEG features are often redundant and contain a large number of redundant features, which will affect the recognition speed and accuracy. Therefore, it is necessary to adopt a feature recursive elimination method, which aims to recursively delete the least important features and select the features that contribute most to the model prediction effect.

[0094] The EEG signal is input, and the computer automatically preprocesses the input EEG signal, and then extracts and selects features from the preprocessed EEG signal. Based on the feature data after feature selection, abnormal brain state diagnosis is performed, and the severity of the abnormality is quantitatively evaluated to end emotion recognition.

[0095] Step 4: Brain state recognition and classification. Execution body: Machine learning SVM algorithm. Description: After feature selection of the extracted feature vectors, the corresponding labels are combined and input into the SVM algorithm model, and the recognition and classification results are output to achieve automatic recognition and classification of anxiety and depression.

[0096] Step 5: Output the results. Execution body: Multi-channel EEG data acquisition and analysis system. Description: After the built-in data automatic preprocessing, differential feature automatic extraction and artificial intelligence recognition algorithm perform data analysis, the results corresponding to the input EEG signal data are obtained, and the degree of possibility is quantitatively evaluated. The results are set in the range of 0-1, that is, the closer to 1, the greater the possibility of illness.

[0097] Using the hardware equipment in this application to collect data from 396 subjects, including 285 normal subjects and 111 anxious subjects, cross-validation was combined with artificial intelligence algorithms (SVM model, RF model, KNN model, XGBoost model, LightGBM model, MLP model), and the emotion recognition prediction results using different input feature types and different artificial intelligence algorithms are shown in Table 1. The emotion recognition prediction results in Table 1 are obtained using Solution Idea 1.

[0098] Table 1 Emotion recognition prediction results of different input feature types and different artificial intelligence algorithms

[0099] It should be noted that the input features in Table 1 use functional connectivity features based on the phase lag index, which is a feasible way to realize EEG features. Other forms of EEG features, such as power spectrum or entropy, may also be used, which is not limited to the present application.

[0100] It can be seen from Table 1 that the accuracy of resting-state EEG features is very low, only 0.692; when using task-state data, the accuracy is significantly improved to 0.961; further using differential EEG features, the accuracy is improved to 0.992. Using task-state-resting-state differential features and state transition differential features as the input of the emotion recognition model, the emotion recognition accuracy is higher.

[0101] This application has the following beneficial effects.

[0102] 1. The portable wearable EEG device adopts a lightweight headband design, 8 forehead collection electrodes, and dry (silver) electrode technology. The overall design is light, comfortable, and collects a lot of information. It can also be worn in other ways to achieve portability.

[0103] 2. In the data collection stage, the two stages of resting state and task state are synchronized to extract the state transition information of the subjects. In the resting state stage, the user remains relaxed to ensure the acquisition of the baseline EEG signal. Subsequently, in the task state stage, the interactive task of answering questions with dynamic pictures is used to greatly stimulate the activities of different areas of the brain, thereby better inducing real EEG waveforms. This not only improves the authenticity of EEG signals, but also greatly improves the accuracy of subsequent algorithms in identifying and analyzing EEG signals. Task stimulation of users can also be achieved through other means such as pictures, videos, audio, and text.

[0104] 3. By using differential EEG features, we can extract the subtle state differences between the resting state and the task state, and between the task states, which can greatly promote the recognition of abnormal emotions.

[0105] The present application also provides an application scenario, which applies the above-mentioned emotion recognition method based on EEG signals. Specifically: The emotion recognition method based on EEG signals provided in this embodiment can be applied in emotion recognition scenarios. The emotion recognition scenario includes a signal acquisition link and an emotion recognition link; the user's resting EEG signals and task-state EEG signals enter the emotion recognition link from the signal acquisition link, and the corresponding emotion recognition prediction results are obtained through human-computer collaboration. The emotion recognition method based on EEG signals provided in this embodiment belongs to the machine labeling link in the emotion recognition link. Specifically, in the emotion recognition link process for the user's resting-state EEG signals and task-state EEG signals, the resting-state EEG signals and the task-state EEG signals can be preprocessed respectively to obtain the preprocessed resting-state EEG signals and the preprocessed task-state EEG signals, and feature extraction is performed on the preprocessed resting-state EEG signals and the preprocessed task-state EEG signals respectively to obtain resting-state EEG features and task-state EEG features, and task-state-resting-state differential features and state transition differential features between resting-state tasks and task-state tasks are calculated based on the resting-state EEG features and task-state EEG features, and the task-state-resting-state differential features and / or state transition differential features after generalized difference processing are input into the trained emotion recognition model to obtain the emotion recognition prediction results of the target user.

[0106] Based on the same inventive concept, the embodiment of the present application also provides an EEG-based emotion recognition device for implementing the EEG-based emotion recognition method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more EEG-based emotion recognition device embodiments provided below can refer to the limitations of the EEG-based emotion recognition method above, and will not be repeated here.

[0107] In an exemplary embodiment, Figure 6As shown, an emotion recognition device based on EEG signals is provided, which includes the following modules.

[0108] The EEG signal acquisition module T1 is used to acquire the resting state EEG signals and task state EEG signals of the target user.

[0109] The preprocessing module T2 is used to preprocess the resting-state EEG signal and the task-state EEG signal respectively to obtain a preprocessed resting-state EEG signal and a preprocessed task-state EEG signal.

[0110] The feature calculation module T3 is used to extract features from the preprocessed resting-state EEG signals and the preprocessed task-state EEG signals, respectively, to obtain resting-state EEG features and task-state EEG features; and to calculate the task-state-resting-state differential features and / or state transition differential features between the resting-state task and the task-state task based on the resting-state EEG features and the task-state EEG features.

[0111] Generalized difference processing module T4 is used based on , perform generalized difference processing on the target features to obtain generalized difference processing features; represents the first system model, is the generalized difference processing feature, Indicates different time periods. is the second system model, is the target feature, which includes the task state-resting state differential feature and / or state transition differential feature between the resting state task and the task state task. For noise.

[0112] The emotion recognition module T5 is used to input the task state-resting state differential features and state transition differential features between the resting state task and the task state task into the trained emotion recognition model to obtain the emotion recognition prediction result of the target user; the trained emotion recognition model is a model trained with the task state-resting state differential features and state transition differential features of the sample user as input and the real emotion recognition results of the sample user as labels.

[0113] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 7As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store emotion recognition data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an emotion recognition method based on electroencephalogram signals is implemented.

[0114] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0115] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0116] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0117] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0118] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0119] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0120] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. An emotion recognition method based on EEG signals, characterized in that: The emotion recognition method based on EEG signals comprises: Obtain the target user's resting-state EEG signals and task-state EEG signals; Preprocessing the resting-state EEG signal and the task-state EEG signal respectively to obtain a preprocessed resting-state EEG signal and a preprocessed task-state EEG signal; Performing feature extraction on the preprocessed resting-state EEG signals and the preprocessed task-state EEG signals respectively to obtain resting-state EEG features and task-state EEG features; calculating task-state-resting-state differential features and / or state transition differential features between resting-state tasks and task-state tasks according to the resting-state EEG features and task-state EEG features; based on , perform generalized difference processing on the target features to obtain generalized difference processing features; represents the first system model, is the generalized difference processing feature, Indicates different time periods. is the second system model, is the target feature, which includes the task state-resting state differential feature and / or state transition differential feature between the resting state task and the task state task. for noise; The generalized difference processing features are input into the trained emotion recognition model to obtain the emotion recognition prediction result of the target user.

2. The method for emotion recognition based on EEG signals according to claim 1, characterized in that: Resting-state EEG signals and task-state EEG signals are EEG signals that include multiple rhythmic dimensions.

3. The method for emotion recognition based on EEG signals according to claim 2, characterized in that: Feature extraction is performed on the preprocessed resting-state EEG signals and the preprocessed task-state EEG signals to obtain resting-state EEG features and task-state EEG features, including: For the preprocessed resting-state EEG signal of each rhythm dimension, calculating the resting-state EEG features of multiple feature index dimensions under the rhythm dimension according to the preprocessed resting-state EEG signal of the rhythm dimension; For each of the resting-state EEG features of each of the characteristic indicator dimensions of the rhythm dimension, segment the resting-state EEG features of the characteristic indicator dimension of the rhythm dimension according to the time length to obtain a number of resting-state segmented features; calculate the mean of all the resting-state segmented features to obtain the resting-state EEG features of the characteristic indicator dimension of the rhythm dimension; The target user performs multiple task-state tasks to obtain task-state EEG signals corresponding to each task-state task; for the preprocessed task-state EEG signals of each rhythm dimension of each task-state task, the task-state EEG features of multiple feature indicator dimensions under the rhythm dimension of the task-state task are calculated according to the preprocessed task-state EEG signals of the rhythm dimension; the task-state EEG features of each feature indicator dimension of each rhythm dimension of each task-state task are used as the task-state EEG features of each feature indicator dimension of each rhythm dimension of each task-state task.

4. The method for emotion recognition based on EEG signals according to claim 3, characterized in that: The task-resting state differential features and state transition differential features between the resting state task and the task-state task are calculated based on the resting state EEG features and the task-state EEG features, including: For each rhythm dimension and each characteristic indicator dimension of each task-state task, subtract the task-state EEG feature and the resting-state EEG feature of the rhythm dimension and the characteristic indicator dimension of the task-state task to obtain the task-state-resting-state differential feature of the rhythm dimension and the characteristic indicator dimension of the task-state task; Determine the state transition differential feature under each of the characteristic indicator dimensions of each of the rhythm dimensions; Feature selection is performed on the task state-resting state differential features of the characteristic indicator dimension of the rhythm dimension of all the task state tasks and all the state transition differential features to obtain the task state-resting state differential features and state transition differential features between the resting state tasks and the task state tasks.

5. The method for emotion recognition based on EEG signals according to claim 4, characterized in that: Determining the state transition differential features under each of the characteristic indicator dimensions of each of the rhythm dimensions specifically includes: Perform differential operation on the task state-resting state differential features of each of the rhythm dimensions and each of the characteristic indicator dimensions of two adjacent task state tasks to obtain the state transition differential features under each of the rhythm dimensions and each of the characteristic indicator dimensions; or, Select M consecutive task-state tasks from all task-state tasks and record them as the first task set; For each of the rhythm dimensions and each of the characteristic indicator dimensions, amortize and merge the task-state EEG features of the rhythm dimension and the characteristic indicator dimension of the M task-state tasks in the first task set to obtain a first merged feature of the rhythm dimension and the characteristic indicator dimension; The next task state task of the first task state task of the selected M task state tasks is taken as the first task state task, and M consecutive task state tasks are selected from all task state tasks again, which are recorded as the second task set; For each of the rhythm dimensions and each of the characteristic indicator dimensions, amortize and merge the task-state EEG features of the rhythm dimension and the characteristic indicator dimension of all task-state tasks in the second task set to obtain a second merged feature of the rhythm dimension and the characteristic indicator dimension; Calculate the correlation between the first task set and the second task set according to the first combined feature and the second combined feature of the feature indicator dimension of the rhythm dimension; The correlation between the first task set and the second task set is amortized to obtain the state transition differential features under each of the rhythm dimensions and each of the feature indicator dimensions.

6. The method for emotion recognition based on EEG signals according to claim 1, characterized in that: Obtain the target user's resting-state EEG signals and task-state EEG signals, including: Acquire a resting state original signal and a task state original signal of a target user; the resting state original signal and the task state original signal are EEG signals collected when the target user performs a resting state task and a task state task respectively; The original signals in the resting state and the original signals in the task state are amplified and filtered respectively to obtain the resting state EEG signals and the task state EEG signals.

7. The method for emotion recognition based on EEG signals according to claim 1, characterized in that: The preprocessing includes downsampling, bandpass filtering, bad segment removal, outlier removal, baseline drift removal and automatic segmentation.

8. An emotion recognition device based on EEG signals, characterized in that: The emotion recognition device based on EEG signals comprises: An EEG signal acquisition module is used to acquire the target user's resting EEG signals and task-state EEG signals; A preprocessing module, used to preprocess the resting-state EEG signal and the task-state EEG signal respectively to obtain a preprocessed resting-state EEG signal and a preprocessed task-state EEG signal; A feature calculation module is used to extract features from the preprocessed resting-state EEG signals and the preprocessed task-state EEG signals, respectively, to obtain resting-state EEG features and task-state EEG features; and to calculate task-state-resting-state differential features and / or state transition differential features between resting-state tasks and task-state tasks according to the resting-state EEG features and task-state EEG features; Generalized difference processing module for , perform generalized difference processing on the target features to obtain generalized difference processing features; represents the first system model, is the generalized difference processing feature, Indicates different time periods. is the second system model, is the target feature, which includes the task state-resting state differential feature and / or state transition differential feature between the resting state task and the task state task. for noise; The emotion recognition module is used to input the generalized difference processing features into the trained emotion recognition model to obtain the emotion recognition prediction result of the target user.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for emotion recognition based on electroencephalogram signals as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for emotion recognition based on electroencephalogram signals described in any one of claims 1 to 7 is implemented.

Images