A wearable emotion monitoring method and system integrating EEG and functional near infrared

Abstract

The present invention relates to an emotion monitoring method and system integrating electroencephalogram (EEG) and functional near infrared. The method comprises: real-time collection of EEG signals and near infrared light density change signals of the left and right prefrontal lobes and temporal lobes of the brain of a subject in a resting state and an induced state; feature extraction of the EEG signals and the near infrared light density change signals to obtain EEG signal features and blood oxygen concentration change signal features, constructing an emotion detection model based on a support vector machine algorithm, and detecting and identifying the negative abnormal emotions of the subject through the emotion detection model based on the EEG signal features and the blood oxygen concentration change signal features. The present invention detects the brain neural electrical activity and blood oxygen metabolism indicators of the brain emotional response by integrating EEG and functional near infrared technology, thereby improving the accuracy of emotion recognition.

Description

Technical Field

[0001] The present invention relates to the field of biomedical information technology, and in particular to a wearable emotion monitoring method and system integrating electroencephalogram and functional near infrared. Background Art

[0002] With the development of the economy and society, people of different ages are frequently experiencing negative emotions such as anxiety and depression due to excessive pressure from work, study and life. Under normal circumstances, negative emotions will gradually ease through certain adjustments or over time. However, emotional disorders such as depression are long-term low moods without cause, which cannot be alleviated even if the external environment changes. People will only go to the hospital for diagnosis and treatment when negative emotions affect their normal work, school and social interactions. Therefore, it is particularly important to detect, regulate and warn of abnormal negative emotions at an early stage.

[0003] Current methods for monitoring emotional changes based on voice, facial expressions, etc. often fail to obtain accurate and objective monitoring results. Emotion monitoring based on physiological signals has gradually gained attention, including functional magnetic resonance imaging, electroencephalogram, heart rate, skin impedance, etc., but there are still some problems: ① Existing methods rely on hospital equipment to obtain collected diagnostic data and analyze them to evaluate the user's emotional state. The detection instruments are often complex and bulky, and cannot achieve real-time wearable monitoring at home. They cannot effectively warn of abnormal emotions that occur over a long period of time. ② Single modal signals cannot effectively characterize the brain's stress response process and characteristics caused by negative emotional changes, cannot effectively regulate short-term negative emotions, and cannot accurately evaluate emotions. Summary of the invention

[0004] The purpose of the present invention is to provide a wearable emotion monitoring method and system integrating electroencephalogram and functional near infrared, which can improve the accuracy of emotion recognition and is easy to use.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] The present invention provides an emotion monitoring method integrating electroencephalogram and functional near infrared, and the emotion monitoring method comprises:

[0007] Real-time collection of EEG signals and near-infrared light density change signals of the left and right prefrontal lobes and temporal lobes of the subject's brain in resting state and induced state; the resting state is the state of the brain when it is awake, eyes closed and relaxed; the induced state is the state of the brain after emotional stimulation is presented to the subject;

[0008] Extracting features of the EEG signal and the near-infrared light density change signal to obtain EEG signal features and blood oxygen concentration change signal features;

[0009] Construct emotion detection model based on support vector machine algorithm;

[0010] Based on the EEG signal characteristics and the blood oxygen concentration change signal characteristics, the negative abnormal emotions of the subject are detected and identified through the emotion detection model.

[0011] Optionally, the emotional stimulus is different types of emotional video clips, the types of the emotional video clips include happiness, relaxation, sadness and fear, and the types of the emotional video clips are determined by the emotional valence and arousal score determined by the universal emotion scale SAM.

[0012] Optionally, before extracting features from the EEG signal and the near-infrared light density change signal to obtain EEG signal features and blood oxygen concentration change signal features, the method further includes: preprocessing the EEG signal and the near-infrared light density change signal respectively;

[0013] The EEG signal is re-referenced, band-pass filtered and baseline corrected, and an independent component analysis method is used to remove artifact interference, and the EEG signal after the artifact interference is removed is segmented to obtain a plurality of EEG signal segments including positive emotions and negative emotions; the positive emotions include happiness and relaxation; the negative emotions include sadness and fear;

[0014] The near-infrared light density change signal is calculated according to the improved Beer-Lambert law to obtain a blood oxygen concentration change signal; the blood oxygen concentration change signal is band-pass filtered and baseline corrected, and the baseline-corrected blood oxygen concentration change signal is segmented to obtain a plurality of blood oxygen concentration change signal segments including positive emotions and negative emotions.

[0015] Optionally, the extracting features of the EEG signal and the near-infrared light density change signal to obtain EEG signal features and blood oxygen concentration change signal features specifically includes:

[0016] Extracting initial features from the EEG signal segment and the blood oxygen concentration change signal segment respectively; the initial features include slope, kurtosis, differential entropy and coherent brain network features;

[0017] The initial features of the EEG signal segment and the blood oxygen concentration change signal segment are respectively merged to obtain EEG signal features and blood oxygen concentration change signal features.

[0018] Optionally, the extraction process of the coherent brain network features specifically includes:

[0019] Dividing the EEG signal segments into a plurality of different frequency bands;

[0020] Merging the EEG signal segments of different frequency bands with the blood oxygen concentration change signal segments to obtain a signal sequence of multiple channels;

[0021] Calculate the coherence matrix between each channel separately;

[0022] Obtaining a maximum value of an average path length of the coherence matrix, and selecting a set threshold value according to the maximum value of the average path length;

[0023] The coherence matrix is ​​binarized according to the set threshold, and network attribute parameters are calculated according to the binarized coherence matrix, and the network attribute parameters are used as coherent brain network features; the network attribute parameters include characteristic path length, clustering coefficient, global efficiency and local efficiency.

[0024] Optionally, after detecting and identifying the negative abnormal emotions of the subject, it also includes: comparing the emotion type results obtained after detection and identification with different types of emotion video clips presented to the subject to determine the emotion state evaluation index; the emotion state evaluation index includes the emotion response accuracy rate, the emotion response negativity rate and the emotion response positivity rate.

[0025] To achieve the above object, the present invention also provides the following solution:

[0026] A wearable emotion monitoring system integrating EEG and functional near infrared comprises: a handheld terminal, an emotion monitoring subsystem and a wearable EEG and near infrared signal acquisition subsystem;

[0027] The handheld terminal is used to present emotional stimulation to the subject to be tested, so that the subject's brain is in an induced state; the induced state is the state of the subject's brain after the emotional stimulation is presented;

[0028] The wearable EEG and near-infrared signal acquisition subsystem covers the prefrontal lobe area and temporal lobe area on the left and right sides of the brain of the subject to be tested, and is used to collect EEG signals and near-infrared light density change signals of the prefrontal lobe and temporal lobe on the left and right sides of the brain of the subject to be tested in real time in a resting state and an induced state;

[0029] The emotion monitoring subsystem is embedded in the handheld terminal and is used to detect and identify the negative abnormal emotions of the subject according to the EEG signal and the near-infrared light density change signal.

[0030] Optionally, the handheld terminal includes:

[0031] The emotional video playing module is used to play different types of emotional video clips to the test subject; the types of emotional video clips include happiness, relaxation, sadness and fear, and the types of emotional video clips are determined by the emotional valence and arousal score determined by the universal emotion scale SAM.

[0032] Optionally, the wearable EEG and near-infrared signal acquisition subsystem includes:

[0033] A supporting headband, worn on the head of the subject;

[0034] EEG electrodes, located on the supporting headband, are used to collect EEG signals from the scalp of the prefrontal lobe and temporal lobe regions on the left and right sides of the brain of the subject;

[0035] A near-infrared emitting diode, located on the supporting headband, for emitting near-infrared light signals to the cerebral scalp of the subject to be tested;

[0036] A near-infrared photosensitive receiving transistor, located on the supporting headband, for receiving the near-infrared light signal;

[0037] The signal processing and transmission module is respectively connected to the EEG electrode, the near-infrared photosensitive receiving transistor and the handheld terminal, and is used to process the EEG signal and the near-infrared light signal to obtain the EEG signal and the near-infrared light density change signal, and send the EEG signal and the near-infrared light density change signal to the handheld terminal.

[0038] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0039] The present invention provides an emotion monitoring method and system integrating EEG and functional near infrared, the method comprising: real-time collection of EEG signals and near infrared light density change signals of the left and right prefrontal lobes and temporal lobes of the brain of a subject in a resting state and an induced state; feature extraction of the EEG signals and the near infrared light density change signals to obtain EEG signal features and blood oxygen concentration change signal features, constructing an emotion detection model based on a support vector machine algorithm, and detecting and identifying the negative abnormal emotions of the subject through the emotion detection model based on the EEG signal features and the blood oxygen concentration change signal features. The present invention detects the brain neural electrical activity and blood oxygen metabolism indicators of the brain emotional response by integrating EEG and functional near infrared technology, which can fully characterize the brain response process and characteristics of emotions and improve the accuracy of emotion recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1 This is a flow chart of a wearable emotion monitoring method integrating EEG and functional near infrared according to the present invention;

[0042] Figure 2 This is a schematic diagram of the emotional state evaluation index of the present invention;

[0043] Figure 3 A schematic diagram of a wearable emotion monitoring system integrating EEG and functional near infrared according to the present invention;

[0044] Figure 4 The working process of a specific embodiment of the present invention;

[0045] Figure 5 This is the overall working process of a specific embodiment of the present invention.

[0046] Explanation of symbols:

[0047] Handheld terminal-1, wearable EEG and near-infrared signal acquisition subsystem-2, supporting headband-21, EEG electrodes-T7, F7, Fp1, Fpz, Fp2, F8, T8, R, near-infrared emitting diode name-S1, S2, near-infrared photosensitive receiving transistors-D1, D2, D3, D4, D5, D6, D7, D8. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] The purpose of the present invention is to provide a wearable emotion monitoring method and system integrating electroencephalogram and functional near infrared, which can improve the accuracy of emotion recognition and is easy to use.

[0050] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] like Figure 1 As shown, the present invention provides an emotion monitoring method integrating EEG and functional near infrared, and the emotion monitoring method comprises:

[0052] S1: Real-time collection of EEG signals and near-infrared light density change signals HbO of the left and right prefrontal lobes and temporal lobes of the subject's brain in resting state and induced state; the resting state is the state of the brain when it is awake, eyes closed and relaxed; the induced state is the state of the brain after emotional stimulation is presented to the subject.

[0053] S2: Extract features of the EEG signal and the near-infrared light density change signal to obtain EEG signal features and blood oxygen concentration change signal features.

[0054] S3: Construct emotion detection model based on support vector machine algorithm.

[0055] S4: Based on the EEG signal characteristics and the blood oxygen concentration change signal characteristics, the negative abnormal emotions of the subject are detected and identified through the emotion detection model.

[0056] Furthermore, in step S1, the emotional stimulus is different types of emotional video clips, and the types of the emotional video clips include happiness, relaxation, sadness and fear. The types of the emotional video clips are determined by the emotional valence and arousal score determined by the universal emotion scale SAM. Specifically, 5 video clips of happiness, relaxation, sadness and fear are randomly played from the constructed emotional video library, each of which is about 30 seconds long, with a 3-second interval between each, and a total length of about 11 minutes, to induce the user's emotions. When each video is played, a synchronous induced marker signal is triggered, and the emotional state is classified and identified through the emotion detection model. Among them, the synchronous induced marker signal is a pulse signal sent to the EEG-functional near-infrared signal acquisition subsystem at the beginning of the video playback, so that the starting time of the induced signal can be found during the subsequent signal processing segmentation.

[0057] The type of the emotional video clip is determined by the emotional valence and arousal score determined by the universal emotion scale SAM, the emotional valence represents a score of 1-9 from negative to positive, and the emotional arousal represents a score of 1-9 from low responsiveness to high responsiveness, wherein an emotional video valence score ≥5 points and an arousal score ≥5 points is judged as happy; an emotional video valence score <5 points and an arousal score ≥5 points is judged as fear; an emotional video valence score <5 points and an arousal score <5 points is judged as sad; an emotional video valence score ≥5 points and an arousal score <5 points is judged as relaxed.

[0058] Preferably, before step S2, the method further includes: preprocessing the EEG signal and the near-infrared light density change signal respectively.

[0059] The EEG signal and the electrode R reference signal are re-referenced, 0.5-49 Hz band-pass filtered, and baseline correction is performed based on the 3 s interval signal. The independent component analysis method is used to remove artifact interference such as electrooculography. The EEG signal after artifact interference is removed is segmented according to the synchronous induced marker signal to obtain 10 EEG signal segments including positive emotions and 10 EEG signal segments including negative emotions. The positive emotions include happiness and relaxation; the negative emotions include sadness and fear.

[0060] The near-infrared light density change signal is calculated according to the improved Beer-Lambert law to obtain a blood oxygen concentration change signal; the blood oxygen concentration change signal is subjected to 0.01-0.2 Hz bandpass filtering, and a baseline correction is performed based on the 3 s interval signal; the baseline-corrected blood oxygen concentration change signal is segmented according to the synchronous induced marker signal to obtain 10 blood oxygen concentration change signal segments including positive emotions and 10 blood oxygen concentration change signal segments including negative emotions.

[0061] Further, step S2: extracting features from the EEG signal and the near-infrared light density change signal to obtain EEG signal features and blood oxygen concentration change signal features specifically includes:

[0062] S21: performing initial feature extraction on the EEG signal segment and the blood oxygen concentration change signal segment respectively; the initial features include slope, kurtosis, differential entropy and coherent brain network features.

[0063] S22: Merging the initial features of the EEG signal segment and the blood oxygen concentration change signal segment respectively to obtain EEG signal features and blood oxygen concentration change signal features.

[0064] The calculation formula of skewness is:

[0065]

[0066] The calculation formula of kurtosis is:

[0067]

[0068] The calculation formula of differential entropy feature DE is:

[0069]

[0070] Where T is the signal sequence length, is the time domain mean of the signal, σ is the standard deviation of the signal sequence, and e is a natural constant.

[0071] Furthermore, in step S21, the extraction process of the coherent brain network features specifically includes:

[0072] The EEG signal segments are divided into a plurality of different frequency bands using a Butterworth bandpass filter, including five different frequency bands: δ (1-4 Hz), θ (4-8 Hz), α (8-13 Hz), β (13-30 Hz) and γ (30-49 Hz).

[0073] The EEG signal segments of different frequency bands are combined with the blood oxygen concentration change signal segments to obtain a signal sequence of multiple channels.

[0074] Calculate the coherence matrix COH between each channel respectively.

[0075] The maximum value of the average path length of the coherence matrix is ​​obtained, and a set threshold is selected according to the maximum value of the average path length.

[0076] The coherence matrix is ​​binarized according to the set threshold, and network attribute parameters are calculated according to the binarized coherence matrix, and the network attribute parameters are used as coherent brain network features; the network attribute parameters include characteristic path length, clustering coefficient, global efficiency and local efficiency.

[0077] The calculation formula of the coherence matrix COH is:

[0078]

[0079] Among them, mean() is the mean calculation function, S i,j is the cross power spectral density of the signals of channels i and j, S i,i and S j,j are the autopower spectral densities of the signals of channels i and j respectively.

[0080] The calculation formula of the characteristic path length L is:

[0081]

[0082] The calculation formula of the clustering coefficient C is:

[0083]

[0084] The calculation formula of the global efficiency E is:

[0085]

[0086] The local efficiency E loc The calculation formula is:

[0087]

[0088] Where N is the number of signal channels, d ij is the shortest path length between nodes i and j, t i is the number of closed triangles adjacent to node i, d jh (N i ) is the shortest path length between nodes j and h that includes neighbor node i, a ij is the connection status of nodes i and j, 1 if connected, 0 otherwise, k i is the degree of node i, d ij -1is the inverse of the shortest path length between nodes i and j, a ih is the connection status between nodes i and h, d jh is the shortest path length between i and h.

[0089] Furthermore, an emotion detection model is constructed based on the support vector machine algorithm. The EEG and HbO feature sets are induced by the video materials of happy and relaxed positive emotions and sad and fear negative emotions, which are used to construct the optimal classification boundary of the support vector machine, so that positive and negative emotions can be accurately classified, and the initialized user emotion monitoring model is obtained. Given the training set T = {(x1, y1),…(x l ,y l )}, where x i ∈R n is the EEG / HbO characteristic signal, y i ∈{-1,1} is the emotion category, -1 represents negativity, 1 represents positivity, i=1,…,l, construct the following optimization problem:

[0090]

[0091] sty i ((w·x i )+b)≥1-ξ i ,

[0092] ξ i ≥0,i=1,…,l

[0093] Where w and b are the normal vector and intercept of the optimal classification surface, respectively, i is the sample point x i The error caused by misclassification, C>0 is the error term ξ i The penalty parameter is obtained by solving the quadratic programming problem. i , then according to

[0094]

[0095] It can be obtained that: when f(x)≥0, the output is 1; when f(x)<0, the output is -1.

[0096] Furthermore, after detecting and identifying the negative abnormal emotions of the subject in step S4, the method further includes: comparing the emotion type result obtained after the detection and identification with different types of emotion video clips presented to the subject to determine an emotion state evaluation index; the emotion state evaluation index includes an emotion response accuracy rate, an emotion response negativity rate, and an emotion response positivity rate, and the emotion response accuracy rate is determined by referring to Figure 2 .

[0097] The calculation formula of the emotional response accuracy is:

[0098]

[0099] The calculation formula of the emotional response negativity rate is:

[0100]

[0101] The calculation formula of the emotional response positivity rate is:

[0102]

[0103] Among them, ACC is the emotion response accuracy rate, NR is the emotion response negativity rate, PR is the emotion response positivity rate, TP is the number of positive emotions responded correctly, TN is the number of negative emotions responded correctly, FP is the number of positive emotions responded as negative, and FN is the number of negative emotions responded as positive.

[0104] Specifically, when the subject is performing emotion monitoring, different emotion videos can be extracted from the constructed emotion video library for viewing to induce emotional response, and the emotion response type can be predicted through the constructed emotion monitoring model, and compared with the extracted emotion video type to calculate the emotion state evaluation index. When the above ACC is high or the NR and PR are low, the emotional response is judged to be normal; when the above ACC is low and the NR is high, the user has a tendency to have a negative emotional response, and the emotional response is judged to be abnormal. The above process can be repeated to re-extract emotional video materials for emotion induction detection. When the user's negative emotional response tendency is monitored multiple times or for a long time, the system will issue an early warning, and at the same time, with the user's consent, the monitoring data will be sent to the cloud server for storage or to designated medical institutions and physicians for further symptom diagnosis.

[0105] To achieve the above purpose, Figure 3 As shown, the present invention also provides a wearable emotion monitoring system integrating EEG and functional near infrared, including: a handheld terminal 1, an emotion monitoring subsystem, and a wearable EEG and near infrared signal acquisition subsystem 2.

[0106] The handheld terminal 1 is used to present emotional stimulation to the subject to be tested, so as to put the subject's brain into an induced state; the induced state is the state of the subject's brain after the emotional stimulation is presented.

[0107] The wearable EEG and near-infrared signal acquisition subsystem 2 covers the prefrontal lobe area and temporal lobe area on the left and right sides of the brain of the subject, and is used to collect EEG signals and near-infrared light density change signals of the prefrontal lobe and temporal lobe on the left and right sides of the brain of the subject in real time in the resting state and induced state.

[0108] The emotion monitoring subsystem is embedded in the handheld terminal 1 and is used to detect and identify the negative abnormal emotions of the subject according to the EEG signal and the near-infrared light density change signal.

[0109] Specifically, the handheld terminal 1 includes: an emotional video playing module, which is used to play different types of emotional video clips to the subject; the types of emotional video clips include happiness, relaxation, sadness and fear, and the types of emotional video clips are determined by the emotional valence and arousal score determined by the universal emotion scale SAM.

[0110] Furthermore, the wearable EEG and near-infrared signal acquisition subsystem 2 includes:

[0111] The support headband 21 is worn on the head of the subject to be tested; both sides of the support headband 21 are retractable Velcro for easy wearing.

[0112] The EEG electrodes are located on the supporting headband 21 and are used to collect EEG signals from the scalp of the left and right frontal lobes and temporal lobes of the subject's brain.

[0113] The near-infrared emitting diode is located on the supporting headband 21 and is used to emit near-infrared light signals to the cerebral scalp of the subject to be tested.

[0114] The near-infrared photosensitive receiving transistor is located on the supporting headband 21 and is used to receive the near-infrared light signal.

[0115] The signal processing and transmission module is connected to the EEG electrode, the near-infrared photosensitive receiving transistor and the handheld terminal 1 respectively, and is used to process the EEG signal and the near-infrared light signal to obtain the EEG signal and the near-infrared light density change signal, and send the EEG signal and the near-infrared light density change signal to the handheld terminal 1. Specifically, the signal transmission part is a Bluetooth communication chip.

[0116] Specifically, in a specific embodiment of the present invention, EEG electrodes, near-infrared photosensitizing receiving transistors and signal processing and transmission modules are encapsulated in a flexible supporting headband, which can collect EEG and blood oxygen concentration change signals in resting and induced states in real time, and connect to a Bluetooth communication chip for wireless information transmission.

[0117] like Figure 3As shown in the figure, 7 Ag / AgCl EEG electrodes correspond to T7, F7, Fp1, Fpz, Fp2, F8 and T8 from left to right according to the international 10-20 system. There is an ear-clip reference electrode R for EEG signal re-reference. S1 and S2 are near-infrared emitters, located between F7 and Fp1, Fp2 and F8 respectively. 8 near-infrared receiving photosensitive transistors Di, i = 1, 2...8 are near-infrared receiving electrodes. S1-D1, S1-D2, S1-D3, S1-D4, S2-D5, S2-D6, S2-D7, S2-D8 constitute an 8-channel brain blood oxygen signal acquisition channel. The interval between each pair of near-infrared emitting and receiving photoelements is 3 cm. S1 and S2 have dual emission wavelengths of 760nm and 850nm. The above-mentioned 8 EEG electrodes are electrically connected to the 8-bit input terminal of the ADS1299 chip in the signal processing module to realize power amplification and analog-to-digital conversion of 8-channel EEG signals. The output signal of the ADS1299 chip is transmitted to the MSP430 microcontroller of the signal processing module through the serial peripheral interface SPI. The MSP430 microcontroller in the above-mentioned signal processing module sends 2 control signals, which are connected to the power controller to control the lighting of the near-infrared emitting diodes S1 and S2. The 8 near-infrared receiving photosensitive transistors perform power amplification and analog-to-digital conversion on the collected near-infrared light density change signal through the power amplifier and the high-speed analog-to-digital converter. The output signal of the high-speed digital-to-analog converter is transmitted to the MSP430 microcontroller of the signal processing and transmission module through the serial communication interface IIC. The MSP430 microcontroller controls the synchronous collection of EEG signals and near-infrared light signals through a real-time clock, and then wirelessly transmits the collected signals to the handheld terminal through the Bluetooth chip for emotion monitoring. The working process of the specific embodiment of the present invention is as follows. Figure 4 and Figure 5 shown.

[0118] Technical effects of the present invention:

[0119] Compared with the current technology of emotion monitoring based on physiological signals, 1) the existing technology only uses single-modal physiological signals, including functional magnetic resonance imaging, electroencephalography, heart rate, skin impedance, etc. to analyze emotional states, which cannot accurately evaluate emotions. Existing research results have fully proved that the brain is the root of emotions. The present invention integrates electroencephalography and functional near-infrared technology to detect brain neural electrical activity and blood oxygen metabolism indicators of brain emotional responses, which can fully characterize the brain's response process and characteristics of emotions and improve the accuracy of emotion recognition.

[0120] 2) The existing technology relies on hospital equipment to obtain collected diagnostic data and analyze it to evaluate the user's emotional state. The detection instruments are often complex and bulky, and cannot achieve real-time monitoring that can be worn at home. It cannot effectively warn of abnormal emotions that appear for a long time. The emotion monitoring headband constructed by the present invention can realize home detection, and provide evaluation indicators of whether the emotional response is normal or not based on the detection results, which is convenient for users to understand intuitively.

[0121] 3) There are already methods for diagnosing depression through emotion detection, which mostly collects resting EEG or functional near-infrared data and analyzes the differences in brain response with normal people. This method uses emotion-inducing materials to evoke the user's emotional response, which can more directly measure the user's attention bias towards different types of emotions and avoid detection errors caused by individual differences.

[0122] Therefore, the present invention evaluates emotional changes by integrating EEG and functional near-infrared to perform wearable real-time monitoring of brain neural electrical activity and blood oxygen metabolism indicators. Users can monitor and regulate short-term negative emotions at home, and warn of long-term negative emotions. The present invention adopts wearable EEG-near-infrared dual-modal detection means to evaluate emotional state in real time, which can be used for long-term negative abnormal emotion detection and warning, and has important significance and value for early screening and diagnosis of emotional disorders such as depression.

[0123] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0124] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A wearable emotion monitoring method integrating EEG and functional near infrared, characterized in that: The emotion monitoring method comprises: Real-time collection of EEG signals and near-infrared light density change signals of the left and right prefrontal lobes and temporal lobes of the subject's brain in resting state and induced state; the resting state is the state of the brain when it is awake, eyes closed and relaxed; the induced state is the state of the brain after emotional stimulation is presented to the subject; The EEG signal is re-referenced, band-pass filtered and baseline corrected, and an independent component analysis method is used to remove artifact interference, and the EEG signal after the artifact interference is removed is segmented to obtain a plurality of EEG signal segments including positive emotions and negative emotions; the positive emotions include happiness and relaxation; the negative emotions include sadness and fear; The near-infrared light density change signal is calculated according to the improved Beer-Lambert law to obtain a blood oxygen concentration change signal; the blood oxygen concentration change signal is band-pass filtered and baseline corrected, and the baseline-corrected blood oxygen concentration change signal is segmented to obtain a plurality of blood oxygen concentration change signal segments including positive emotions and negative emotions; Performing initial feature extraction on the EEG signal segment and the blood oxygen concentration change signal segment respectively; the initial features include slope, kurtosis, differential entropy and coherent brain network features; Merging the initial features of the EEG signal segment and the blood oxygen concentration change signal segment respectively to obtain EEG signal features and blood oxygen concentration change signal features; Construct emotion detection model based on support vector machine algorithm; Based on the EEG signal characteristics and the blood oxygen concentration change signal characteristics, detecting and identifying the negative abnormal emotions of the subject through the emotion detection model; The extraction process of the coherent brain network features specifically includes: Dividing the EEG signal segments into a plurality of different frequency bands; Merging the EEG signal segments of different frequency bands with the blood oxygen concentration change signal segments to obtain a signal sequence of multiple channels; Calculate the coherence matrix between each channel separately; Obtaining a maximum value of an average path length of the coherence matrix, and selecting a set threshold value according to the maximum value of the average path length; The coherence matrix is ​​binarized according to the set threshold, and network attribute parameters are calculated according to the binarized coherence matrix, and the network attribute parameters are used as coherent brain network features; the network attribute parameters include characteristic path length, clustering coefficient, global efficiency and local efficiency.

2. The wearable emotion monitoring method integrating EEG and functional near infrared according to claim 1 is characterized in that: The emotional stimuli are different types of emotional video clips, and the types of the emotional video clips include happiness, relaxation, sadness and fear. The types of the emotional video clips are determined by the emotional valence and arousal score determined by the universal emotion scale SAM.

3. The wearable emotion monitoring method integrating EEG and functional near infrared according to claim 1 is characterized in that: After detecting and identifying the negative abnormal emotions of the subject, it also includes: comparing the emotion type results obtained after detection and identification with different types of emotion video clips presented to the subject to determine the emotion state evaluation index; the emotion state evaluation index includes the emotion response accuracy rate, the emotion response negativity rate and the emotion response positivity rate.

4. A wearable emotion monitoring system integrating EEG and functional near infrared, characterized in that: The wearable emotion monitoring system integrating EEG and functional near infrared is based on the wearable emotion monitoring method integrating EEG and functional near infrared according to any one of claims 1 to 3, comprising: a handheld terminal, an emotion monitoring subsystem, and a wearable EEG and near infrared signal acquisition subsystem; The handheld terminal is used to present emotional stimulation to the subject to be tested, so that the subject's brain is in an induced state; the induced state is the state of the subject's brain after the emotional stimulation is presented; The wearable EEG and near-infrared signal acquisition subsystem covers the prefrontal lobe area and temporal lobe area on the left and right sides of the brain of the subject to be tested, and is used to collect EEG signals and near-infrared light density change signals of the prefrontal lobe and temporal lobe on the left and right sides of the brain of the subject to be tested in real time in a resting state and an induced state; The emotion monitoring subsystem is embedded in the handheld terminal and is used to detect and identify the negative abnormal emotions of the subject according to the EEG signal and the near-infrared light density change signal.

5. The wearable emotion monitoring system integrating EEG and functional near infrared according to claim 4 is characterized in that: The handheld terminal comprises: The emotional video playing module is used to play different types of emotional video clips to the test subject; the types of emotional video clips include happiness, relaxation, sadness and fear, and the types of emotional video clips are determined by the emotional valence and arousal score determined by the universal emotion scale SAM.

6. The wearable emotion monitoring system integrating EEG and functional near infrared according to claim 4 is characterized in that: The wearable EEG and near-infrared signal acquisition subsystem includes: A supporting headband, worn on the head of the subject; EEG electrodes, located on the supporting headband, are used to collect EEG signals from the scalp of the prefrontal lobe and temporal lobe regions on the left and right sides of the brain of the subject; A near-infrared emitting diode, located on the supporting headband, for emitting near-infrared light signals to the cerebral scalp of the subject to be tested; A near-infrared photosensitive receiving transistor, located on the supporting headband, for receiving the near-infrared light signal; The signal processing and transmission module is respectively connected to the EEG electrode, the near-infrared photosensitive receiving transistor and the handheld terminal, and is used to process the EEG signal and the near-infrared light signal to obtain the EEG signal and the near-infrared light density change signal, and send the EEG signal and the near-infrared light density change signal to the handheld terminal.

Images