A brain-computer interface system based on multifunctional emotion recognition and adaptive regulation

By introducing emotion-induced units, EEG signal acquisition module and stress signal emotion calibration module in the brain-computer interface system, combining the space-frequency optimization algorithm and feature fusion model, the problem of low accuracy of emotion recognition in the existing technology is solved, and the Efficient emotion recognition and regulation improves the accuracy of the system and user experience.

CN113778228BActive Publication Date: 2024-12-03HARBIN INST OF TECH SHENZHEN GRADUATE SCHOOL
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111062641.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-10
Publication Date
2024-12-03
Estimated Expiration
2041-09-10

AI Technical Summary

Technical Problem

The existing brain-computer interface system has low accuracy in emotional recognition, and the single EEG signal characteristics are insufficient, making it difficult to effectively identify and regulate different emotions.

Method used

A brain-computer interface system for multifunctional emotion recognition and adaptive regulation is designed, including an emotion-induced unit, an EEG signal acquisition module, a stress signal emotion calibration module, an emotion decoding unit and an emotion regulation unit, through the space frequency optimization algorithm and stress Signal characteristics are fused, combined with pressure sensors for emotional decoding and adjustment, and realize adaptive learning function.

Benefits of technology

The accuracy of emotion recognition is improved, and the pressure sensor is used as an emotional catharsis tool to achieve multiple effects of emotion regulation and calibration, enhancing the system's control accuracy and user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113778228B_ABST
    Figure CN113778228B_ABST
Patent Text Reader

Abstract

The present invention provides a brain-computer interface system based on multifunctional emotion recognition and adaptive regulation, including an emotion induction unit, an electroencephalogram signal acquisition module, a pressure signal emotion calibration module, an emotion decoding unit and an emotion regulation unit. The beneficial effects of the present invention are: (1) the space-frequency optimization method is used to realize the extraction of electroencephalogram features under different emotions and combine the pressure signal for feature fusion to improve the accuracy of emotion recognition. (2) the emotion tagger is changed to a pressure sensor type, and the pressure emotion signal is added without increasing the test, so that the emotion recognition rate is improved. At the same time, the pressure sensor can be used as an emotion catharsis tool in the measurement to achieve multiple functions such as emotion regulation, emotion calibration, and emotion measurement.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to human-computer interaction, emotion recognition and emotion regulation technologies, and in particular to a brain-computer interface system based on multifunctional emotion recognition and adaptive regulation. Background Art

[0002] Emotion recognition refers to identifying people's emotional changes under external stimuli, and is widely used in the fields of psychological state detection, crime prevention, disease treatment, etc. In recent years, EEG signals for emotion recognition have gradually attracted attention, especially in the fields of psychological state detection and emotion regulation, which has important practical application value.

[0003] Since EEG signals are a direct response to brain electrical signals, they have more intuitive performance than other signals such as electrocardiogram and skin electricity that rely on sympathetic and parasympathetic nerve regulation. For example, EEG signals have a wide range of spatial-frequency distribution differences under different emotions, so EEG signals have better emotion classification and recognition effects. Existing studies have revealed the spatial-frequency differences of EEG signals under different emotions. The research of Fumikazu et al. shows that the activity intensity of theta waves in the frontal lobe increases in the mental state, while the a wave activity in the occipital lobe is suppressed. On the contrary, in the resting state, the research of WL Zheng et al. shows that positive emotions have higher energy in beta and gamma waves than negative emotions, and positive emotions have stronger energy in the frontal lobe in beta and gamma waves, while the activity energy of delta waves in the parietal lobe and occipital lobe is enhanced under negative emotions. Based on this, the spatial-frequency optimization method can be used to extract EEG features under different emotions to achieve the classification of different emotions.

[0004] The brain generates signals with certain characteristics to communicate and control electronic devices (such as computers, etc.), which is called a brain-computer interface (BCI). The key to the brain-computer interface is to identify the characteristics of brain signals and convert them into correct commands to control devices. By performing emotion recognition on EEG and applying the results of emotion recognition to the computer's reverse regulation of emotions, a brain-computer interface system based on emotion recognition can be realized. However, the accuracy of single EEG emotion recognition is not high enough, and the information used for emotion calibration is not sufficient. Summary of the invention

[0005] The present invention provides a brain-computer interface system based on multifunctional emotion recognition and adaptive regulation, characterized in that it comprises an emotion inducing unit, an electroencephalogram signal acquisition module, a pressure signal emotion calibration module, an emotion decoding unit and an emotion regulation unit.

[0006] The emotion inducing unit is used to play audio and / or video emotion inducing materials to induce different emotions of the tester;

[0007] The EEG signal acquisition module is used to acquire EEG signals under different emotional states;

[0008] The pressure signal emotion calibration module includes a first pressure sensor, a second pressure sensor, and a third pressure sensor. The first to third pressure sensors are used to record the pressure of pressing the pressure sensors under different emotions when the tester watches / listens to emotion-inducing materials during the training phase, as a basis for auxiliary emotion discrimination;

[0009] The emotion decoding unit is used for emotion decoding;

[0010] The emotion regulation unit is used to regulate negative emotions and calm down excited emotions.

[0011] As a further improvement of the present invention, the pressure signal emotion calibration module also includes a fourth pressure sensor, which serves as a detector for knocking in the discrimination stage. The fourth pressure sensor serves as an emotion signal collector and also as a vent for anxiety and excitement, which has a reverse effect on emotion regulation.

[0012] As a further improvement of the present invention, the emotion-inducing material includes positive, neutral and negative emotion materials in the Chinese emotion stimulation material library, and the positive, neutral and negative emotion materials are played to induce the positive, neutral and negative emotions of the tester, and the EEG signals corresponding to the emotions are collected through the EEG signal collection module.

[0013] As a further improvement of the present invention, the first to third pressure sensors are respectively labeled as "positive", "neutral" and "negative", and the first to third pressure sensors record the pressure under the corresponding emotion-inducing materials. Since the pressure of the pressure sensors under different emotions is different, the first to third pressure sensors serve as emotion calibration and also as emotion discrimination marks.

[0014] As a further improvement of the present invention, the emotion decoding unit performs emotion decoding by using a scale-weighted space-frequency optimization algorithm and pressure signal emotion discrimination, and then performs feature fusion.

[0015] As a further improvement of the present invention, the emotion regulation unit enables the brain-computer interface system to have an adaptive learning function, and can adjust parameters according to the environment, time and individual users.

[0016] As a further improvement of the present invention, the method for performing emotion decoding by using a space-frequency optimization algorithm is as follows:

[0017] Step 1: Perform a 1-45 Hz bandpass filter on the EEG signal to retain the EEG signal in the delta to gamma wave frequency band;

[0018] Step 2: Divide the passband, and then perform a common spatial pattern on each subband to calculate its projection matrix;

[0019] Step 3: Combine the scale scores of the training set, optimize the subspace projection matrix, perform weighted averaging, and form a new projection matrix; classify the test data to find the optimal projection matrix and the optimal classification subband;

[0020] Step 4: Classify the EEG signal using the optimal sub-band and the trained optimal projection matrix to obtain the features generated by the EEG signal.

[0021] As a further improvement of the present invention, in the emotion decoding unit, feature fusion is completed by a feature fusion model, and the feature fusion model is as follows:

[0022]

[0023] Where d is the decoding dimension of the system, f d×1 is the output, i.e., the result of the fusion of the space-frequency optimization algorithm and the stress signal emotion discrimination feature, M is the number of system signals, Represents the eigenvector of the i-th signal, with N as the number of features i , yes The weight is d×N i The matrix of d×1 is a d×1 vector, which is the bias coefficient;

[0024] Use softmax to implement task decision:

[0025]

[0026] Among them, f k,1 is the vector f d×1 The value with subscript (k,1).

[0027] As a further improvement of the present invention, the current emotional state is obtained through feature fusion and discrimination, and the emotional regulation unit implements emotional regulation in the following manner: if multiple tests show negative emotions, the emotional induction unit plays positive emotional materials to induce positive emotions; if multiple tests show positive emotions, the emotional induction unit plays neutral emotional materials to calm the emotions;

[0028] During the emotion recognition process, the tester can press the fourth pressure sensor at will. The pressure sensor serves as an emotion signal collector and can also be used as a vent for anxiety and excitement, which in turn affects emotion regulation.

[0029] The beneficial effects of the present invention are as follows: (1) The space-frequency optimization method is used to extract EEG features under different emotions and combine them with pressure signals for feature fusion to improve the accuracy of emotion recognition. (2) The emotion tagger is changed to a pressure sensor type, and the pressure emotion signal is added without increasing the test, so that the emotion recognition rate is improved. At the same time, the pressure sensor can be used as an emotion catharsis tool in the measurement to achieve multiple functions such as emotion regulation, emotion calibration, and emotion measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a principle block diagram of the brain-computer interface system of the present invention;

[0031] Figure 2 is a schematic diagram of a stress signal emotion calibration module;

[0032] Figure 3 It is a schematic diagram of the emotion space-frequency optimization discrimination algorithm;

[0033] Figure 4 It is a schematic diagram of the stress signal emotion discrimination method;

[0034] Figure 5 This is a block diagram of the adaptive learning structure based on softmax regression;

[0035] Figure 6 It is a schematic diagram of the emotion regulation unit realizing emotion regulation. DETAILED DESCRIPTION

[0036] like Figure 1 As shown, the present invention discloses a brain-computer interface system based on multifunctional emotion recognition and adaptive regulation, including an EEG signal acquisition module, a pressure signal emotion calibration module, an emotion induction unit, an emotion decoding unit, an emotion regulation unit, and an adaptive learning module. The EEG signal acquisition module collects EEG signals under different emotional states; the pressure signal emotion calibration module includes four pressure sensors, the first three of which are used in the training stage to record the pressure of pressing the sensor under different emotions when watching the induced video, as a basis for auxiliary emotion discrimination, and the fourth is used as a detector for tapping in the discrimination stage; the emotion induction unit plays audio, video and other emotion inducing materials; the emotion decoding unit performs emotion decoding through a time-space-frequency optimization algorithm; the emotion regulation unit realizes functions such as regulating negative emotions and calming down excited emotions; the adaptive learning module is used for users to learn before using the system for the first time, perform the set behavioral tasks according to the operating instructions, collect physiological activity signals during the execution of the tasks, and calculate the best parameters for feature extraction and task classification.

[0037] The schematic diagram of the stress signal emotion calibration module is as follows Figure 2As shown, the emotion induction unit stimulates the corresponding emotions of the tester, and the pressure signal emotion calibration module includes four recessed pressure sensors, three of which are marked with "positive", "neutral" and "negative", and the fourth is marked with "Please press after measurement". In the emotion training stage, after watching the induced materials played by the emotion induction unit, the tester presses the pressure sensor marked with "positive" after watching the positive stimulating materials, presses the pressure sensor marked with "neutral" after watching the neutral stimulating materials, and presses the pressure sensor marked with "negative" after watching the negative stimulating materials. The force is carried out according to the corresponding emotion. The pressure under the corresponding induced material is recorded by the pressure sensor. Since the pressure of the pressure sensor is different under different emotions, the pressure sensor is used as an emotion calibration and also as a discrimination mark of emotion.

[0038] During the emotion training phase, testers need to fill out the valence-arousal scale to make detailed annotations of their emotions.

[0039] The emotion regulation unit enables the brain-computer interface system to have adaptive learning capabilities, and can adjust parameters according to the environment, time and individual users, thereby ensuring the accuracy of system control to the greatest extent.

[0040] The schematic diagram of the space-frequency optimization algorithm is as follows Figure 3 As shown in the figure, the EEG signal is bandpass filtered at 1-45 Hz, and the EEG signal in the delta wave to gamma wave frequency band is retained. The passband is divided, and then the common space mode is performed on each sub-band to calculate its projection matrix. Combined with the scale score of the training set, the subspace projection matrix is ​​optimized, and a weighted average is performed to form a new projection matrix. The test data is classified to find the optimal projection matrix and the optimal classification sub-band. The EEG signal is classified by the optimal sub-band and the best trained projection matrix to obtain the features generated by the EEG signal.

[0041] Stress signal emotion identification method Figure 4 As shown, in the emotion training stage, after watching the inducing materials played by the emotion inducing unit, the tester presses the pressure sensor marked with "positive" after watching the positive stimulating materials, presses the pressure sensor marked with "neutral" after watching the neutral stimulating materials, and presses the pressure sensor marked with "negative" after watching the negative stimulating materials, and the strength is according to the corresponding emotion.

[0042] Figure 5 Shows a simplified learning model for softmax regression.

[0043] The emotion regulation unit implements emotion regulation such as Figure 6As shown in the figure, the current emotional state (positive, neutral, negative) is obtained through feature fusion and discrimination. For multiple tests with negative emotions, positive emotional materials are played through the emotion induction unit to induce positive emotions. For multiple tests with positive emotions, neutral emotional materials are played through the emotion induction unit to calm the emotions. In the emotion recognition stage, the tester can press the fourth pressure sensor at will. The pressure sensor acts as an emotional signal collector, and can also be used as a catharsis device for anxiety and excitement, which has a counter-effect on emotional regulation.

[0044] In view of the shortcomings of the existing technology, the present invention designs the emotion calibration module into a pressure signal emotion calibration module, collects the pressure and the emotion label when different emotions are induced; at the same time, an emotion space-frequency optimization algorithm is designed to decode emotions, and a feature fusion model is used to fuse pressure features and EEG space-frequency features to optimize the discrimination of emotions. Combined with brain-computer interface technology, emotions are reversely regulated. For multiple tests with negative emotions, positive emotions are induced by playing positive emotion materials through the emotion induction unit. For multiple tests with positive emotions, neutral emotion materials are played through the emotion induction unit to calm emotions.

[0045] The beneficial effects of the present invention are:

[0046] (1) The space-frequency optimization method is used to extract EEG features under different emotions and combine them with stress signals for feature fusion to improve the accuracy of emotion recognition.

[0047] (2) The emotion tagger is changed to a pressure sensor type. Without increasing the number of tests, the pressure emotion signal is added to improve the emotion recognition rate. At the same time, the pressure sensor can be used as an emotion catharsis tool in the measurement to achieve multiple functions such as emotion regulation, emotion calibration, and emotion measurement.

[0048] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A brain-computer interface system based on multifunctional emotion recognition and adaptive regulation, characterized in that: It includes an emotion induction unit, an EEG signal acquisition module, a pressure signal emotion calibration module, an emotion decoding unit and an emotion regulation unit. The emotion inducing unit is used to play audio and / or video emotion inducing materials to induce different emotions of the tester; The EEG signal acquisition module is used to acquire EEG signals under different emotional states; The pressure signal emotion calibration module includes a first pressure sensor, a second pressure sensor, and a third pressure sensor. The first to third pressure sensors are used to record the pressure of pressing the pressure sensors under different emotions when the tester watches / listens to emotion-inducing materials during the training phase, as a basis for auxiliary emotion discrimination; The emotion decoding unit is used for emotion decoding; The emotion regulation unit is used to regulate negative emotions and calm down excited emotions; The emotion decoding unit uses a scale weighted space-frequency optimization algorithm and pressure signal emotion discrimination, and then performs feature fusion to perform emotion decoding; The method of emotion decoding through space-frequency optimization algorithm is as follows: Step 1: Perform a 1-45 Hz bandpass filter on the EEG signal to retain the EEG signal in the delta to gamma wave frequency band; Step 2: Divide the passband, and then perform a common spatial pattern on each subband to calculate its projection matrix; Step 3: Combine the scale scores of the training set, optimize the subspace projection matrix, perform weighted averaging, and form a new projection matrix; classify the test data to find the optimal projection matrix and the optimal classification subband; Step 4: Classify the EEG signal by the optimal sub-band and the trained optimal projection matrix to obtain the features generated by the EEG signal; In the emotion decoding unit, feature fusion is accomplished through a feature fusion model, which is as follows: Where d is the decoding dimension of the system, f d×1 is the output, i.e., the result of the fusion of the space-frequency optimization algorithm and the stress signal emotion discrimination feature, M is the number of system signals, Represents the eigenvector of the i-th signal, with N as the number of features i , yes The weight is d×N i The matrix of d×1 is a d×1 vector, which is the bias coefficient; Use softmax to implement task decision: Among them, f k,1 is the vector f d×1 The value with subscript (k,1).

2. The brain-computer interface system according to claim 1, characterized in that: The pressure signal emotion calibration module also includes a fourth pressure sensor, which serves as a detector for knocking in the discrimination stage. The fourth pressure sensor serves as an emotion signal collector and also as a catharsis device for anxiety and excitement, which has a reverse effect on emotion regulation.

3. The brain-computer interface system according to claim 1, characterized in that: The emotion-inducing materials include positive, neutral and negative emotion materials in the Chinese emotion stimulation material library. The positive, neutral and negative emotion materials are played to induce the positive, neutral and negative emotions of the tester, and the EEG signals corresponding to the emotions are collected through the EEG signal collection module.

4. The brain-computer interface system according to claim 1, characterized in that: The first to third pressure sensors are respectively marked as "positive", "neutral" and "negative". The first to third pressure sensors record the pressure under the corresponding emotion-inducing materials. Since the pressure of the pressure sensors is different under different emotions, the first to third pressure sensors are used as emotion calibration and also as emotion discrimination marks.

5. The brain-computer interface system according to claim 1, characterized in that: The emotion regulation unit enables the brain-computer interface system to have an adaptive learning function, and can adjust parameters according to the environment, time and individual users.

6. The brain-computer interface system according to claim 1, characterized in that: The current emotional state is determined by feature fusion. The emotional regulation unit implements emotional regulation in the following way: for multiple tests with negative emotions, the emotional induction unit plays positive emotional materials to induce positive emotions; For those who have positive emotions in multiple tests, neutral emotional materials are played through the emotion induction unit to calm the emotions; During the emotion recognition process, the tester can press the fourth pressure sensor at will. The pressure sensor serves as an emotion signal collector and can also be used as a vent for anxiety and excitement, which in turn affects emotion regulation.