An emotion recognition method, device, equipment and medium based on electroencephalogram signals

Abstract

The present application discloses an emotion recognition method, device, equipment and medium based on electroencephalogram signals, relating to the technical field of electroencephalogram signal emotion recognition. The method includes: respectively preprocessing the resting-state electroencephalogram signals and the task-state electroencephalogram signals to obtain the preprocessed resting-state electroencephalogram signals and the preprocessed task-state electroencephalogram signals; respectively extracting features from the preprocessed resting-state electroencephalogram signals and the preprocessed task-state electroencephalogram signals to obtain resting-state electroencephalogram features and task-state electroencephalogram features; calculating the task-state - resting-state differential features and / or state transition differential features between the resting state and the task state according to the resting-state electroencephalogram features and the task-state electroencephalogram features; inputting the task-state - resting-state differential features and / or state transition differential features after generalized differential processing into a trained emotion recognition model to obtain the emotion recognition prediction result of the target user. The present application improves the accuracy of emotion recognition.

Description

Technical Field

[0001] The present application relates to the technical field of electroencephalogram signal emotion recognition, and particularly to an emotion recognition method, device, equipment and medium based on electroencephalogram signals. Background Art

[0002] Mild abnormal emotions such as depression and anxiety are difficult to detect and diagnose. People often regard the initial symptoms as stress, fatigue or temporary dissatisfaction in life. Due to these symptoms being easily misunderstood, combined with factors such as time and energy, people often expect to confront them through subjective will or fear discrimination from others when they feel unwell, and do not seek professional help in time, thus delaying diagnosis and subsequent treatment. The diagnosis of abnormal emotions such as depression and anxiety mainly relies on subjective scales and doctor consultations, which have problems such as strong subjectivity and low accuracy. Therefore, there is an urgent need for an emotion recognition method with high accuracy. Summary of the Invention

[0003] The purpose of the present application is to provide an emotion recognition method, device, equipment and medium based on electroencephalogram signals, which can improve the accuracy of emotion recognition.

[0004] To achieve the above purpose, the present application provides the following solutions:

[0005] In the first aspect, the present application provides an emotion recognition method based on electroencephalogram signals, including the following steps.

[0006] Obtain the resting-state electroencephalogram signal and task-state electroencephalogram signal of the target user.

[0007] Preprocess the resting-state electroencephalogram signal and the task-state electroencephalogram signal respectively to obtain the preprocessed resting-state electroencephalogram signal and the preprocessed task-state electroencephalogram signal.

[0008] Extract features from the preprocessed resting-state electroencephalogram signal and the preprocessed task-state electroencephalogram signal respectively to obtain resting-state electroencephalogram features and task-state electroencephalogram features; calculate the task-state - resting-state differential features and / or state transition differential features between the resting-state task and the task-state task according to the resting-state electroencephalogram features and the task-state electroencephalogram features.

[0009] Based on , perform generalized differential processing on the target feature to obtain the generalized differential processing feature; represents the first system model, is the generalized differential processing feature, represents different time periods, is the second system model, is the target feature, and the target feature includes the task-state - resting-state differential features and / or state transition differential features between the resting-state task and the task-state task, is noise.

[0010] Input the generalized difference processing feature into the trained emotion recognition model to obtain the emotion recognition prediction result of the target user.

[0011] In a second aspect, the present application provides an emotion recognition device based on electroencephalogram signals, including the following modules.

[0012] An electroencephalogram signal acquisition module for acquiring the resting-state electroencephalogram signal and task-state electroencephalogram signal of a target user.

[0013] A preprocessing module for preprocessing the resting-state electroencephalogram signal and the task-state electroencephalogram signal respectively to obtain the preprocessed resting-state electroencephalogram signal and the preprocessed task-state electroencephalogram signal.

[0014] A feature calculation module for extracting features from the preprocessed resting-state electroencephalogram signal and the preprocessed task-state electroencephalogram signal respectively to obtain resting-state electroencephalogram features and task-state electroencephalogram features; calculating the task-state - resting-state difference feature and / or state transition difference feature between the resting-state task and the task-state task according to the resting-state electroencephalogram features and the task-state electroencephalogram features.

[0015] A generalized difference processing module for performing generalized difference processing on the target feature to obtain the generalized difference processing feature; represents the first system model, is the generalized difference processing feature, represents different time periods, is the second system model, is the target feature, and the target feature includes the task-state - resting-state difference feature and / or state transition difference feature between the resting-state task and the task-state task, is noise.

[0016] An emotion recognition module for inputting the generalized difference processing feature into the trained emotion recognition model to obtain the emotion recognition prediction result of the target user.

[0017] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the computer program to implement the above-mentioned emotion recognition method based on electroencephalogram signals.

[0018] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned emotion recognition method based on electroencephalogram signals is implemented.

[0019] According to the specific embodiments provided in the present application, the following technical effects are disclosed in the present application: The present application provides an emotion recognition method, device, equipment and medium based on electroencephalogram (EEG) signals. After obtaining the preprocessed resting-state EEG signals and the preprocessed task-state EEG signals, feature extraction is respectively performed on the preprocessed resting-state EEG signals and the preprocessed task-state EEG signals to obtain resting-state EEG features and task-state EEG features. According to the resting-state EEG features and the task-state EEG features, the task-state - resting-state differential features and / or state transition differential features between the resting-state task and the task-state task are calculated. At least one of the task-state - resting-state differential features and the state transition differential features is subjected to generalized difference processing, and the generalized difference processed features are used as the input of the emotion recognition model for emotion recognition. A machine learning model is used to learn the law between the generalized difference processed task-state - resting-state differential features and / or state transition differential features and the clear recognition results of the user. The trained emotion recognition model is used for emotion recognition, avoiding the subjective influence of humans and being more accurate. Among them, the EEG features are non-stationary and non-linear time series data. By using the generalized difference processing method to process the task-state - resting-state differential features and / or state transition differential features, it can have stronger adaptability and prediction ability, thereby further improving the emotion recognition prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0021] Figure 1 It is an application environment diagram of an emotion recognition method based on EEG signals in an embodiment of the present application.

[0022] Figure 2 It is a flowchart of an emotion recognition method based on EEG signals provided in an embodiment of the present application.

[0023] Figure 3 It is a schematic diagram of the specific process of an emotion recognition method based on EEG signals provided in an embodiment of the present application.

[0024] Figure 4 It is a schematic diagram of the functional design of the hardware device provided in an embodiment of the present application.

[0025] Figure 5 It is a schematic diagram of the preprocessing process provided in an embodiment of the present application.

[0026] Figure 6Schematic diagram of functional modules of an emotion recognition device based on electroencephalogram (EEG) signals provided by an embodiment of the present application.

[0027] Figure 7 Schematic diagram of the structure of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0028] Next, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0029] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0030] The emotion recognition method based on EEG signals provided by the embodiments of the present application can be applied to an application environment as shown in Figure 1 . Among them, the terminal 102 communicates with the server 104 through a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set separately, integrated on the server 104, placed in the cloud or on other servers. The terminal 102 can send the resting-state EEG signals and task-state EEG signals of the target user to the server 104. After receiving the resting-state EEG signals and task-state EEG signals of the target user, the server 104 preprocesses the resting-state EEG signals and task-state EEG signals respectively to obtain the preprocessed resting-state EEG signals and preprocessed task-state EEG signals, extracts features from the preprocessed resting-state EEG signals and preprocessed task-state EEG signals respectively to obtain resting-state EEG features and task-state EEG features, calculates the task-state - resting-state differential features and state transition differential features between the resting-state task and the task-state task according to the resting-state EEG features and task-state EEG features, and inputs the task-state - resting-state differential features and / or state transition differential features after generalized differential processing into the trained emotion recognition model to obtain the emotion recognition prediction result of the target user. The server 104 can feedback the obtained emotion recognition prediction result for the target user to the terminal 102. In addition, in some embodiments, the emotion recognition method based on EEG signals can also be implemented by the server 104 or the terminal 102 alone. For example, the terminal 102 can directly perform emotion recognition on the resting-state EEG signals and task-state EEG signals, or the server 104 can obtain the resting-state EEG signals and task-state EEG signals from the data storage system and perform emotion recognition on the resting-state EEG signals and task-state EEG signals.

[0031] Among them, the terminal 102 can be but is not limited to various desktop computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers, and can also be a cloud server.

[0032] In an exemplary embodiment, as Figure 2 shown, a method for emotion recognition based on electroencephalogram (EEG) signals is provided. This method is executed by a computer device, and specifically can be executed alone by a computer device such as a terminal or a server, or can be jointly executed by a terminal and a server. In the embodiments of the present application, taking this method applied to Figure 1 the server 104 in

[0033] Step 201, obtain the resting-state EEG signal and task-state EEG signal of the target user.

[0034] Step 202, preprocess the resting-state EEG signal and the task-state EEG signal respectively to obtain the preprocessed resting-state EEG signal and the preprocessed task-state EEG signal.

[0035] Step 203, extract features from the preprocessed resting-state EEG signal and the preprocessed task-state EEG signal respectively to obtain the resting-state EEG features and task-state EEG features; calculate the task-state - resting-state differential features and / or state transition differential features between the resting-state task and the task-state task according to the resting-state EEG features and task-state EEG features.

[0036] Step 204, based on , perform a generalized difference process on the target feature to obtain the generalized difference process feature; represents the first system model, is the generalized difference process feature, represents different time periods, is the second system model, is the target feature, and the target feature includes the task-state - resting-state differential features and / or state transition differential features between the resting-state task and the task-state task, is the noise.

[0037] Step 205: Input the generalized difference processing feature into the trained emotion recognition model to obtain the emotion recognition prediction result of the target user. In this embodiment, the emotion recognition model can have two meanings: 1. Emotion judgment, such as normal emotion and abnormal emotion; 2. Quantitative assessment of the severity level, the severity level of abnormal emotion. Then the emotion recognition prediction result can include the emotion judgment result of the user and the severity level (likelihood level) of the abnormal emotion. The trained emotion recognition model is a model trained with the generalized difference processing feature of the sample user as the input and the true emotion recognition result of the sample user as the label.

[0038] Implementing the above steps 201 to 205, using the task-state - resting-state difference feature and the state transition difference feature as the input of the emotion recognition model for emotion recognition improves the accuracy of emotion recognition. By using a machine learning model to learn the law between the task-state - resting-state difference feature and the state transition difference feature and the clear recognition result of the user, and using the trained emotion recognition model for emotion recognition, it avoids the subjective influence of humans and is more accurate. Aiming at the problems in the related technology such as few leads, no task, and low accuracy, this application provides a lightweight brain emotion detection device and method to achieve accurate recognition of abnormal emotions. The technical solution of this application has technical advantages such as a high number of leads, a design of combining resting-state and task-state emotion induction, and differential electroencephalogram features.

[0039] The above step 201 may include the following steps 301 to 302.

[0040] Step 301: Obtain the resting-state original signal and the task-state original signal of the target user; the resting-state original signal and the task-state original signal are electroencephalogram signals collected when the target user performs the resting-state task and the task-state task respectively. The resting state is usually the state with eyes open and no task, and the task state is usually the state when the subject receives external stimuli; the external stimuli can be pictures, music, texts, videos, animations, etc.; the task state can be multiple stimulus tasks connected together with a consistent time length.

[0041] Step 302: Amplify and filter the resting-state original signal and the task-state original signal respectively to obtain the resting-state electroencephalogram signal and the task-state electroencephalogram signal.

[0042] The signal acquisition of the emotion recognition method based on electroencephalogram signals provided by this application is implemented based on a hardware device, and the function design of the hardware device is as Figure 4 shown. The hardware device includes an eight-lead silver electrode, an analog front end, a microcontroller, a Bluetooth module, a computer terminal, a storage module, and a lithium battery.

[0043] The physical design of the signal acquisition module is mainly divided into two parts. The first part is the hardware of the data host: the analog front-end part is implemented with the ADS1299-8 chip to amplify and filter the raw resting-state signals and raw task-state signals; the microcontroller part is implemented with the STM32F103RCT6 chip; the communication part is completed by the HC42 Bluetooth module, and the data processed by the analog front-end is transmitted to the computer, and the computer is equipped with supporting software to display the data processed by the analog front-end; the power supply part is completed by a lithium battery with a battery voltage of 3.7V to provide power for the analog front-end, the microcontroller, and the Bluetooth module; the storage module part is completed by the built-in 128M memory. The second part is the signal acquisition electrode part: 8 silver dry electrodes are placed as active electrodes according to the 10-20 system, and at the same time, 1 earlobe grounding electrode is set as the reference electrode. The acquisition device fixes the EEG activity electrodes on the non-hair-covered area of the forehead through elastic fabric, or fixes them on the forehead through a rigid shell combined with a rotatable button. The data host is fixed on the elastic fabric or the rigid shell.

[0044] Software design of the multi-channel EEG data acquisition system: Design a supporting software based on Python and Java languages to analyze and visualize the acquired EEG data and generate analysis results. There are mainly the following functional modules: First, waveform display: Under this functional module, the EEG signal data of the specified channel and the corresponding recording switch can be visualized in real time, and the filtering range and filter order can be adjusted. At the same time, functions such as fast Fourier transform, offline data playback, data forwarding, and ordinary EEG signal data acquisition are provided. Second, data acquisition: Under this functional module, EEG data acquisition is carried out under the induced state. Emotional induction is carried out through emotional stimulation gifs, and scales are synchronously acquired to obtain EEG signals and scale data. Third, result presentation: Based on the built-in feature extraction algorithm and artificial intelligence algorithm, this functional module identifies the abnormal emotion types of the subjects (i.e., users), such as anxiety and depression. Fourth, user management: Under this functional module, user information is managed. Fifth, data management: Manage the acquired EEG signal data. Sixth, connection setting. Under this functional module, an effective connection between the acquisition device and the computer is set.

[0045] Signal acquisition paradigm design: Under the waveform display module, the user only needs to wear the acquisition device and ensure that the acquisition device is normally connected to the computer, then the user can collect online EEG data and save the EEG signal data through the software. Under the data acquisition module, the brain state recognition for depression and anxiety can be carried out. The EEG signal data acquisition is designed in two steps: First, set a resting state task with a duration of 30s. The user must concentrate on staring at a crosshair pattern in the center of the computer monitor. Second, based on the Self-Rating Depression Scale (SDS) and the Self-Rating Anxiety Scale (SAS), design 40 questionnaire questions of emotional induction dynamic pictures in the task state. The answering time for each question should be no less than 5s, and only one answer can be submitted for each question. The options cannot be modified after submission.

[0046] The resting-state EEG signals and task-state EEG signals include EEG signals in multiple rhythm dimensions, and the rhythm dimensions can include rhythm, rhythm, rhythm, rhythm and rhythm. Based on the above acquisition process, the resting-state EEG signals with a time length of 30 seconds in each rhythm dimension of the target user and the task-state EEG signals in each rhythm dimension corresponding to 40 task states can be obtained.

[0047] In another exemplary embodiment of the present application, as Figure 5 shown, the preprocessing of the resting-state EEG signals and task-state EEG signals obtained in step 201 sequentially includes downsampling, band-pass filtering, removing bad segments, removing outliers, removing baseline drift, and automatic segmentation to prepare for subsequent feature extraction and selection.

[0048] In the above step 203, feature extraction is respectively performed on the preprocessed resting-state EEG signals and the preprocessed task-state EEG signals to obtain resting-state EEG features and task-state EEG features, which specifically include the following steps 401 to 403.

[0049] Step 401: For the preprocessed resting-state EEG signals in each rhythm dimension, calculate the resting-state EEG features in multiple feature index dimensions in the rhythm dimension according to the preprocessed resting-state EEG signals in the rhythm dimension.

[0050] Step 402: For the resting-state EEG features of each of the rhythm dimensions and each of the characteristic index dimensions, segment them according to the time length of the resting-state EEG features of the rhythm dimension and the characteristic index dimension to obtain a number of resting-state segmented features; calculate the mean of all the resting-state segmented features to obtain the resting-state EEG features of the rhythm dimension and the characteristic index dimension.

[0051] Step 403: The target user performs multiple task-state tasks to obtain the task-state EEG signals corresponding to each task-state task; for the preprocessed task-state EEG signals of each rhythm dimension of each task-state task, calculate the task-state EEG features of multiple characteristic index dimensions under the rhythm dimension of the task-state task; take the task-state EEG features of each task-state task, each rhythm dimension, and each characteristic index dimension as the task-state EEG features of each task-state task, each rhythm dimension, and each characteristic index dimension.

[0052] Select the phase lag index (PLI), power spectrum, or entropy in the brain-brain EEG features as the recognition features, and calculate the multi-dimensional EEG feature data through the formula. The calculation formula of the phase lag index is shown in Equation (1).

[0053] (1);

[0054] In the formula, represents the th column and the th column of the phase lag index of the EEG signals. The EEG signals are all in matrix form, and the column represents the lead signals between two electrodes. There are eight electrodes set in this application, so this matrix has 7 columns; represents the th column and the th column of the instantaneous phase difference of the EEG time series signals at the moment, is the sign function, represents taking the average, represents taking the absolute value. The value of PLI is between 0 and 1. When PLI is 0, it means that there is no phase synchronization effect between the two columns of EEG signals. When PLI is 1, it means that the two columns of EEG signals are phase-locked. Finally, the EEG signals are converted into recognizable feature vectors. Synchronously extract the differential EEG features between the resting state and the task state, and between the task states.

[0055] This application proposes a trinity multi-paradigm fusion EEG differential feature extraction technology system of "dynamic baseline correction - cross-paradigm coupling - interpretability mining". Based on EEG data from two paradigms, namely the resting state (open-eye baseline) and the task state (multi-segment emotion-induced animations), EEG differential features of resting state-task state and task state-task state fusion are constructed, which can not only achieve efficient extraction of EEG features but also improve the interpretability of the features.

[0056] (1) Zero-mean normalization to eliminate individual baseline drift: In neuroscience, EEG signals have two paradigms, the resting state and the task state. 1) Resting state features: 30 seconds of resting state, divided into 6 segments of resting state segment features, each segment being 5s, and then averaging to obtain a set of resting state EEG features. 2) Task state features: The EEG features corresponding to each task, and the features of all tasks are tiled and concatenated. The target user performs multiple task state tasks to obtain the task state EEG signals corresponding to each task state task; for the preprocessed task state EEG signals of each rhythm dimension of each task state task, task state feature values of multiple feature index dimensions in the rhythm dimension of the task state task are calculated according to the preprocessed task state EEG signals of the rhythm dimension. 3) Task state - resting state differential features: The task state EEG features corresponding to each task minus the resting state EEG features.

[0057] The resting state EEG features can be denoted as , where is the number of electrodes, each element of represents the eigenvalue of brain region and in the resting state. The task state EEG features can be denoted as , where represents the th task state, and the element represents the eigenvalue of brain region and and in the task state. The resting state contains the individual's inherent neural activity baseline (i.e., , where is zero-mean fluctuation). The task state can be decomposed into the superposition of task-induced changes and the baseline , that is, . When directly using or for classification, the baselines and will mask the task-induced specific changes , through task - resting state zero - mean normalization, the task - resting state differential features are represented as shown in Equation (2) below.

[0058] (2).

[0059] If it is assumed that the individual baseline is consistent in the task and resting states (i.e., , here the resting state needs to be the open - eye resting state), then the task - resting state differential features after zero - mean normalization are represented as Equation (3) below.

[0060] (3).

[0061] The above - mentioned proof process shows that after zero - mean normalization, the baseline drift of the individual can be eliminated, the relative task - induced changes can be retained, and the specific representation of features for emotions can be enhanced.

[0062] For different rhythm dimensions, the task - resting state differential features of all rhythm dimensions can be tiled and connected to obtain the task - resting state differential features finally input into the trained emotion recognition model.

[0063] (2) Differential method for suppressing common - mode noise and low - frequency interference: Differencing adjacent task data can eliminate non - stationary trends. In continuous task states, environmental noise or physiological artifacts (such as eye movements, head movements, breathing) have low - frequency common - mode characteristics. Define the continuous task - state difference, that is, the state - transition differential feature can be expressed as Equation (4) below.

[0064] (4).

[0065] Expand Equation (4) to Equation (5) below.

[0066] (5).

[0067] Among them, is the observation noise of task . Due to the common - mode noise , the power of the noise term is significantly reduced after differencing. At the same time, the difference of the task - induced change can reflect the dynamic neural response pattern, and patients show abnormalities (such as reduced flexibility) in such dynamic characteristics, thus enhancing the classification discriminability.

[0068] For different rhythm dimensions, the state - transition differential features of all rhythm dimensions can be tiled and connected to obtain the state - transition differential features finally input into the trained emotion recognition model.

[0069] The state - transition differential features can be calculated by two methods.

[0070] Method 1: Based on the task state - resting state features, perform differential processing by subtracting the features of the latter task from the former task between two adjacent tasks (i.e., ).

[0071] Method 2: Average and then merge the task - state EEG features of the k - th to n - th task - state tasks into respectively, and then average and merge the PLI features of the (k + 1) - th to (n + 1) - th task - state tasks into , , that is, , and finally average A to obtain the state - transition differential features.

[0072] (3) Mathematical formalization: Generalized differential model of EEG features: Assume that the EEG features follow the system equation shown in Equation (6) below.

[0073] (6).

[0074] Among them, represents different time periods, is the unit backward - shift operator (i.e., ), are model parameters, is the polynomial order. is the first - order system model, which characterizes the autoregressive property of EEG features, that is, how the feature value at the current time period depends on its historical values . In the resting state or task state, the features between brain regions may have time dependence (such as the inertia or persistence of neural activities). . The coefficient of quantifies the strength of this dependence. For example, being larger indicates that the state at the previous moment has a significant impact on the current state, which may correspond to the stability of the neural network. are model parameters, is the polynomial order. is the second - order system model, which characterizes the dynamic driving effect of task input on EEG features, that is, how the external task stimulus affects the feature value through different time delays. Task stimuli may trigger an instantaneous response ( term) or a delayed response ( term) between brain regions. For example, indicates that the task input has an immediate regulatory effect on EEG features, which may correspond to the rapid allocation of attentional resources; This indicates that there is a first-order delay in the regulatory effect, which may reflect the conduction time of nerve signals. is the observed value of EEG features, is the task input stimulus, is the noise. Directly using will introduce baseline deviation . By applying the difference operator to both sides of the equation, as shown in Equation (7).

[0075] (7).

[0076] At this time and have had their means removed, and is white noise with zero mean. In the experiment, the task input is the designed task stimulus, and its difference can be regarded as a pulse excitation, while extracts the task-induced transient neural response, which is more sensitive to EEG feature abnormalities related to depression. The above model can correspond to different tasks by adjusting the order of the system, making the difference EEG features of the output have specific neurobiological meanings and improving the interpretability of this method. The model can be applied through the following two ideas, that is, the generalized difference processing features in Step 204 can be obtained through any one of the following two ideas.

[0077] Solution idea 1: Assume that and are both of zero order, that is, Then the target feature can be regarded as the observed value of EEG features , as the input of the subsequent emotion recognition model.

[0078] Solution idea 2: If and are not of zero order, regard the task-state - resting-state difference feature or state transition difference feature between the resting-state task and the task-state task as an external task stimulus, and obtain the feature when the classification task performance is optimal through a data-driven method. Regard it as and , and then solve the parameters of by least squares fitting, and further obtain the mathematical expression of the generalized difference model of EEG features as the input of the subsequent emotion recognition model. The feature

[0079] when the classification task performance is optimal can be obtained through feature selection methods. It can be used alone or in combination after splicing. That is, any one of the task-state - resting-state differential features and state transition differential features between the resting-state task and the task-state task can be subjected to generalized difference processing and used as the input of the trained emotion recognition model. Or the task-state - resting-state differential features and state transition differential features can be combined, and the combined features can be subjected to generalized difference processing and used as the input of the trained emotion recognition model. The features with the optimal classification task performance can be selected through feature selection methods. As the input of the emotion recognition model.

[0080] Then in the above step 203, calculating the task-state - resting-state differential features and state transition differential features between the resting-state task and the task-state task according to the resting-state EEG features and task-state EEG features specifically includes the following steps 501 to 503.

[0081] Step 501: For each task-state task, each of the rhythm dimensions, and each of the feature index dimensions, subtract the task-state EEG features of the task-state task in the rhythm dimension and the feature index dimension from the resting-state EEG features to obtain the task-state - resting-state differential features of the task-state task in the rhythm dimension and the feature index dimension.

[0082] Step 502: Determine the state transition differential features under each of the rhythm dimensions and each of the feature index dimensions.

[0083] Step 503: Perform feature selection on the task-state - resting-state differential features of all the task-state tasks in the rhythm dimension and the feature index dimension and all the state transition differential features to obtain the task-state - resting-state differential features and state transition differential features between the resting-state task and the task-state task.

[0084] The above step 502 can be implemented in any one of the following two ways.

[0085] (1) The first way: Perform a difference operation on the task-state - resting-state differential features of each of the rhythm dimensions and each of the feature index dimensions for two adjacent task-state tasks to obtain the state transition differential features under each of the rhythm dimensions and each of the feature index dimensions.

[0086] (2) The second way includes the following steps:

[0087] Select M consecutive task-state tasks from all the task-state tasks, denoted as the first task set; M is less than Y, where Y is the total number of task-state tasks;

[0088] For each of the rhythm dimensions and each of the characteristic indicator dimensions, amortize and merge the task-state EEG features of the rhythm dimension and the characteristic indicator dimension of the M task-state tasks in the first task set to obtain a first merged feature of the rhythm dimension and the characteristic indicator dimension;

[0089] The next task state task of the first task state task of the selected M task state tasks is taken as the first task state task, and M consecutive task state tasks are selected from all task state tasks again, which are recorded as the second task set;

[0090] For each of the rhythm dimensions and each of the characteristic indicator dimensions, amortize and merge the task-state EEG features of the rhythm dimension and the characteristic indicator dimension of all task-state tasks in the second task set to obtain a second merged feature of the rhythm dimension and the characteristic indicator dimension;

[0091] Calculate the correlation between the first task set and the second task set according to the first combined feature and the second combined feature of the feature indicator dimension of the rhythm dimension;

[0092] The correlation between the first task set and the second task set is amortized to obtain the state transition differential features under each of the rhythm dimensions and each of the feature indicator dimensions.

[0093] Multidimensional EEG features often contain many redundant and irrelevant features, including more redundant features, which will affect the recognition speed and accuracy. Therefore, it is necessary to adopt feature selection methods to delete unimportant features and retain features that contribute more to the model. The recursive feature elimination (RFE) method can be used for feature selection.

[0094] The emotion recognition module in the present application can be a SVM model, a RF model, a KNN model, an XGBoost model, a LightGBM model or an MLP model, etc.

[0095] The trained emotion recognition model is a model trained with the task-resting state differential features and state transition differential features of sample users as input and the real emotion recognition results of sample users as labels. After obtaining the trained emotion recognition model, the task-resting state differential features and state transition differential features between the target user's resting state task and task-state task are input into the trained emotion recognition model to obtain the target user's emotion recognition prediction results. After algorithm recognition processing, a probability value between 0 and 1 will be output, which is positively correlated with the risk of abnormal emotions, that is, the closer the value is to 1, the greater the possibility of illness.

[0096] In the SVM model of the machine learning-based algorithm, the feature data selected in (3) is input into the SVM model to judge and identify the input EEG signals.

[0097] As Figure 3 shown, the emotion recognition method based on EEG signals provided by this application includes the following steps 1 to 5.

[0098] Step 1: EEG signal acquisition. Execution subject: Brain state detection device and multi-channel EEG data acquisition system software. Description: The brain state detection device collects data in a normal state through the waveform display function module, and collects resting state and task state data for depression and anxiety recognition through the data acquisition function module. An analog front-end chip is used to amplify and filter the collected signals to obtain high-quality original EEG signals, providing a data basis for subsequent processing. A wearable structure design is used to improve the portability and comfort during wearing. During the diagnosis and recognition of depression and anxiety, two stages of EEG data acquisition in the resting state and task state are set to obtain brain state data at different stages.

[0099] Step 2: Signal preprocessing. Execution subject: Microcontroller and preprocessing algorithm. Description: The data simply processed by amplification and filtering by the analog front-end is downsampled, band-pass filtered, bad segments removed, outliers removed, baseline drift removed, and automatically segmented to further improve the signal quality and prepare for subsequent feature extraction.

[0100] Step 3: Feature extraction and selection. Execution subject: Feature extraction and selection algorithm. Description: The phase lag index (PLI), power spectrum, or entropy in EEG features is used as the recognition feature. The PLI feature is used to quantify the asymmetry of phase lag or lead between two signals, and at the same time can avoid the volume effect and is used to study the phase synchronization between different brain regions. Through the mathematical calculation formula (1), this feature is calculated and extracted from the preprocessed EEG signals.

[0101] Multidimensional EEG features often have redundancy, which contains many redundant features, thus affecting the recognition speed and accuracy. Therefore, a feature recursive elimination method needs to be adopted, aiming to recursively delete the least important features to select the features that contribute the most to the model prediction effect.

[0102] Input the EEG signals, and the computer automatically preprocesses the input EEG signals, then extracts and selects features from the preprocessed EEG signals, diagnoses brain state abnormalities based on the feature data after feature selection, and quantitatively evaluates the severity of the abnormalities to end the emotion recognition.

[0103] Step 4: Brain state recognition and classification. Execution entity: Machine learning SVM algorithm. Description: After performing feature selection on the extracted feature vectors, the corresponding labels are combined and input into the SVM algorithm model to output the recognition and classification results, realizing the automatic recognition and classification of anxiety and depression.

[0104] Step 5: Output results. Execution entity: Multichannel electroencephalogram data acquisition and analysis system. Description: After performing data automatic preprocessing, differential feature automatic extraction, and artificial intelligence recognition algorithms for data analysis, the results corresponding to the input electroencephalogram signal data are obtained, and a quantitative evaluation of the degree of possibility is carried out. The results are set within the range of 0 - 1, that is, the closer to 1, the greater the possibility of being ill.

[0105] Using the 396 subject data collected by the hardware device in this application, including 285 normal subjects and 111 anxiety subjects, through cross - validation combined with artificial intelligence algorithms (SVM model, RF model, KNN model, XGBoost model, LightGBM model, MLP model), the emotion recognition prediction results with different input feature types and different artificial intelligence algorithms are shown in Table 1. The emotion recognition prediction results in Table 1 are obtained using solution idea 1.

[0106] Table 1 Emotion recognition prediction results with different input feature types and different artificial intelligence algorithms

[0107]

[0108] It should be noted that the input features in Table 1 adopt the functional connection features based on the phase lag index, which is a realizable way of electroencephalogram features. Other forms of electroencephalogram features, such as power spectrum or entropy, can also be used. This application is not limited here.

[0109] It can be seen from Table 1 that the accuracy of the resting - state electroencephalogram features is very low, only 0.692; when using task - state data, the accuracy is significantly improved, reaching 0.961; further using differential electroencephalogram features, the accuracy is increased to 0.992. Using task - state - resting - state differential features and state - transition differential features as the input of the emotion recognition model, the emotion recognition accuracy is higher.

[0110] This application has the following beneficial effects.

[0111] 1. The portable wearable electroencephalogram device adopts a lightweight head - ring design, 8 frontal acquisition electrodes, and dry (silver) electrode technology. The overall design is lightweight, comfortable, and can collect a lot of information. Other wearable methods can also be used to achieve portability.

[0112] 2. During the data acquisition stage, synchronize the resting state and task state to extract the subject's state transition information. In the resting state, the user maintains a relaxed state to ensure obtaining baseline EEG signals. Subsequently, in the task state, use a dynamic graph answering interactive task to greatly stimulate the activities of different brain regions, thereby better inducing real EEG waveforms. This not only improves the authenticity of EEG signals but also significantly enhances the accuracy of subsequent algorithms in identifying and analyzing EEG signals. Task stimuli for the user can also be achieved through other means such as pictures, videos, audio, and text.

[0113] 3. By adopting differential EEG features, it is possible to extract the subtle state differences between the resting state and the task state, as well as between task states, which can greatly facilitate the recognition of abnormal emotions.

[0114] This application also provides an application scenario that applies the above-mentioned emotion recognition method based on EEG signals. Specifically: The emotion recognition method based on EEG signals provided in this embodiment can be applied in an emotion recognition scenario. The emotion recognition scenario includes a signal acquisition link and an emotion recognition link; the user's resting state EEG signals and task state EEG signals enter the emotion recognition link from the signal acquisition link, and corresponding emotion recognition prediction results are obtained through a human-machine collaboration method. The emotion recognition method based on EEG signals provided in this embodiment belongs to the machine marking link in the emotion recognition link. Specifically, during the emotion recognition link process for the user's resting state EEG signals and task state EEG signals, the resting state EEG signals and task state EEG signals can be preprocessed respectively to obtain the preprocessed resting state EEG signals and preprocessed task state EEG signals, and feature extraction is performed on the preprocessed resting state EEG signals and preprocessed task state EEG signals respectively to obtain resting state EEG features and task state EEG features. The task state-resting state differential features and state transition differential features between the resting state task and the task state task are calculated based on the resting state EEG features and task state EEG features, and the task state-resting state differential features and / or state transition differential features after generalized differential processing are input into the trained emotion recognition model to obtain the emotion recognition prediction results of the target user.

[0115] Based on the same inventive concept, the embodiments of this application also provide an emotion recognition device based on EEG signals for implementing the above-mentioned emotion recognition method based on EEG signals. The implementation solutions provided by this device to solve problems are similar to those described in the above method. Therefore, the specific limitations in one or more of the following embodiments of the emotion recognition device based on EEG signals can refer to the limitations on the emotion recognition method based on EEG signals in the above text, and will not be elaborated here.

[0116] In an exemplary embodiment, such as Figure 6As shown, an emotion recognition device based on electroencephalogram (EEG) signals includes the following modules.

[0117] An EEG signal acquisition module T1 for acquiring the resting-state EEG signals and task-state EEG signals of a target user.

[0118] A preprocessing module T2 for preprocessing the resting-state EEG signals and the task-state EEG signals respectively to obtain preprocessed resting-state EEG signals and preprocessed task-state EEG signals.

[0119] A feature calculation module T3 for extracting features from the preprocessed resting-state EEG signals and preprocessed task-state EEG signals respectively to obtain resting-state EEG features and task-state EEG features; calculating the task-state - resting-state differential features and / or state transition differential features between the resting-state tasks and task-state tasks based on the resting-state EEG features and task-state EEG features.

[0120] A generalized difference processing module T4 for performing generalized difference processing on a target feature based on to obtain a generalized difference processed feature; represents the first system model, is the generalized difference processed feature, represents different time periods, is the second system model, is the target feature, and the target feature includes the task-state - resting-state differential features and / or state transition differential features between the resting-state tasks and task-state tasks, is noise.

[0121] An emotion recognition module T5 for inputting the task-state - resting-state differential features and state transition differential features between the resting-state tasks and task-state tasks into a trained emotion recognition model to obtain the emotion recognition prediction result of the target user; the trained emotion recognition model is a model trained with the task-state - resting-state differential features and state transition differential features of a sample user as inputs and the true emotion recognition results of the sample user as labels.

[0122] In an exemplary embodiment, a computer device is provided. The computer device can be a server or a terminal, and its internal structure diagram can be as shown in Figure 7As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store emotion recognition data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. When the computer program is executed by the processor, it implements an emotion recognition method based on electroencephalogram signals.

[0123] Those skilled in the art can understand that Figure 7 the structure shown in the figure is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0124] In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0125] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0126] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0127] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the various embodiments provided in this application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0128] The databases involved in the various embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the various embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., without limitation.

[0129] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0130] Specific examples are used in this article to elaborate on the principles and implementation manners of this application. The descriptions of the above embodiments are only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, according to the idea of this application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. An emotion recognition method based on EEG signals, characterized in that: The emotion recognition method based on EEG signals comprises: Obtain the target user's resting-state EEG signals and task-state EEG signals; Preprocessing the resting-state EEG signal and the task-state EEG signal respectively to obtain a preprocessed resting-state EEG signal and a preprocessed task-state EEG signal; Performing feature extraction on the preprocessed resting-state EEG signals and the preprocessed task-state EEG signals respectively to obtain resting-state EEG features and task-state EEG features; calculating task-state-resting-state differential features and / or state transition differential features between resting-state tasks and task-state tasks according to the resting-state EEG features and task-state EEG features; based on , perform generalized difference processing on the target features to obtain generalized difference processing features; represents the first system model, is the generalized difference processing feature, Indicates different time periods. is the second system model, is the target feature, which includes the task state-resting state differential feature and / or state transition differential feature between the resting state task and the task state task. for noise; The generalized difference processing features are input into the trained emotion recognition model to obtain the emotion recognition prediction result of the target user.

2. The method for emotion recognition based on EEG signals according to claim 1, characterized in that: Resting-state EEG signals and task-state EEG signals are EEG signals that include multiple rhythmic dimensions.

3. The method for emotion recognition based on EEG signals according to claim 2, characterized in that: Feature extraction is performed on the preprocessed resting-state EEG signals and the preprocessed task-state EEG signals to obtain resting-state EEG features and task-state EEG features, including: For the preprocessed resting-state EEG signal of each rhythm dimension, calculating the resting-state EEG features of multiple feature index dimensions under the rhythm dimension according to the preprocessed resting-state EEG signal of the rhythm dimension; For each of the resting-state EEG features of each of the characteristic indicator dimensions of the rhythm dimension, segment the resting-state EEG features of the characteristic indicator dimension of the rhythm dimension according to the time length to obtain a number of resting-state segmented features; calculate the mean of all the resting-state segmented features to obtain the resting-state EEG features of the characteristic indicator dimension of the rhythm dimension; The target user performs multiple task-state tasks to obtain task-state EEG signals corresponding to each task-state task; for the preprocessed task-state EEG signals of each rhythm dimension of each task-state task, the task-state EEG features of multiple feature indicator dimensions under the rhythm dimension of the task-state task are calculated according to the preprocessed task-state EEG signals of the rhythm dimension; the task-state EEG features of each feature indicator dimension of each rhythm dimension of each task-state task are used as the task-state EEG features of each feature indicator dimension of each rhythm dimension of each task-state task.

4. The method for emotion recognition based on EEG signals according to claim 3, characterized in that: The task-resting state differential features and state transition differential features between the resting state task and the task-state task are calculated based on the resting state EEG features and the task-state EEG features, including: For each rhythm dimension and each characteristic indicator dimension of each task-state task, subtract the task-state EEG feature and the resting-state EEG feature of the rhythm dimension and the characteristic indicator dimension of the task-state task to obtain the task-state-resting-state differential feature of the rhythm dimension and the characteristic indicator dimension of the task-state task; Determine the state transition differential feature under each of the characteristic indicator dimensions of each of the rhythm dimensions; Feature selection is performed on the task state-resting state differential features of the characteristic indicator dimension of the rhythm dimension of all the task state tasks and all the state transition differential features to obtain the task state-resting state differential features and state transition differential features between the resting state tasks and the task state tasks.

5. The method for emotion recognition based on EEG signals according to claim 4, characterized in that: Determining the state transition differential features under each of the characteristic indicator dimensions of each of the rhythm dimensions specifically includes: Perform differential operation on the task state-resting state differential features of each of the rhythm dimensions and each of the characteristic indicator dimensions of two adjacent task state tasks to obtain the state transition differential features under each of the rhythm dimensions and each of the characteristic indicator dimensions; or, Select M consecutive task-state tasks from all task-state tasks and record them as the first task set; For each of the rhythm dimensions and each of the characteristic indicator dimensions, amortize and merge the task-state EEG features of the rhythm dimension and the characteristic indicator dimension of the M task-state tasks in the first task set to obtain a first merged feature of the rhythm dimension and the characteristic indicator dimension; The next task state task of the first task state task of the selected M task state tasks is taken as the first task state task, and M consecutive task state tasks are selected from all task state tasks again, which are recorded as the second task set; For each of the rhythm dimensions and each of the characteristic indicator dimensions, amortize and merge the task-state EEG features of the rhythm dimension and the characteristic indicator dimension of all task-state tasks in the second task set to obtain a second merged feature of the rhythm dimension and the characteristic indicator dimension; Calculate the correlation between the first task set and the second task set according to the first combined feature and the second combined feature of the feature indicator dimension of the rhythm dimension; The correlation between the first task set and the second task set is amortized to obtain the state transition differential features under each of the rhythm dimensions and each of the feature indicator dimensions.

6. The method for emotion recognition based on EEG signals according to claim 1, characterized in that: Obtain the target user's resting-state EEG signals and task-state EEG signals, including: Acquire a resting state original signal and a task state original signal of a target user; the resting state original signal and the task state original signal are EEG signals collected when the target user performs a resting state task and a task state task respectively; The original signals in the resting state and the original signals in the task state are amplified and filtered respectively to obtain the resting state EEG signals and the task state EEG signals.

7. The method for emotion recognition based on EEG signals according to claim 1, characterized in that: The preprocessing includes downsampling, bandpass filtering, bad segment removal, outlier removal, baseline drift removal and automatic segmentation.

8. An emotion recognition device based on EEG signals, characterized in that: The emotion recognition device based on EEG signals comprises: An EEG signal acquisition module is used to acquire the target user's resting EEG signals and task-state EEG signals; A preprocessing module, used to preprocess the resting-state EEG signal and the task-state EEG signal respectively to obtain a preprocessed resting-state EEG signal and a preprocessed task-state EEG signal; A feature calculation module is used to extract features from the preprocessed resting-state EEG signals and the preprocessed task-state EEG signals, respectively, to obtain resting-state EEG features and task-state EEG features; and to calculate task-state-resting-state differential features and / or state transition differential features between resting-state tasks and task-state tasks according to the resting-state EEG features and task-state EEG features; Generalized difference processing module for , perform generalized difference processing on the target features to obtain generalized difference processing features; represents the first system model, is the generalized difference processing feature, Indicates different time periods. is the second system model, is the target feature, which includes the task state-resting state differential feature and / or state transition differential feature between the resting state task and the task state task. for noise; The emotion recognition module is used to input the generalized difference processing features into the trained emotion recognition model to obtain the emotion recognition prediction result of the target user.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for emotion recognition based on electroencephalogram signals as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for emotion recognition based on electroencephalogram signals described in any one of claims 1 to 7 is implemented.

Images