Method, device and storage medium for recording emotional fluctuation events

Abstract

This invention provides a method for recording emotional fluctuation events, a device for recording emotional fluctuation events, and a computer-readable storage medium. The method for recording emotional fluctuation events includes the following steps: acquiring the electroencephalogram (EEG) signal of a user to be recorded; determining, based on the EEG signal, a probability value representing that the user's emotion originates from a virtual event; and, in response to the probability value being less than a preset first threshold, determining that the user's emotional fluctuation is caused by a real event, and recording the event content of the real event. This invention can distinguish between emotional fluctuations caused by virtual and real events by judging the source of the user's emotion, and dynamically calculates the intensity of the user's emotion through multimodal signals, used to record the event content of the real event that caused the user's emotional fluctuation.

Claims

1. A method for recording emotional fluctuation events, comprising the following steps: acquiring an electroencephalogram (EEG) signal, an environmental signal, a facial expression signal, and a physiological signal of a user to be recorded; determining a probability value, based on the EEG signal, that the user's emotion originates from a virtual event; and determining that the user's emotional fluctuation is caused by a real event when the probability value is less than a preset first threshold; determining an emotion intensity value, based on the EEG signal, the environmental signal, the facial expression signal, and the physiological signal, that represents the intensity of the user's emotion; and triggering a first instruction to photograph the user to record the event content of the real event when the emotion intensity value is greater than a preset second threshold and the duration of the fluctuation is greater than a preset third threshold.

2. The recording method as described in claim 1, wherein the electroencephalogram (EEG) signal includes waves, waves, waves, waves, and waves, and the step of determining, based on the EEG signal, a probability value representing that the emotion of the user to be recorded originates from a virtual event includes: The method involves using filters to remove noise signals from each of the stated EEG signals, wherein the noise signals include at least power frequency interference and electromyographic noise generated during power system operation; performing frequency domain feature analysis on each of the filtered EEG signals to obtain the power spectral density of each EEG signal, thereby extracting a plurality of feature values ​​corresponding to the wave, the wave, the wave, the wave, and the wave; and determining the probability value based on these feature values ​​of each of the stated EEG signals.

3. The recording method as described in claim 1, wherein the step of determining the probability value further includes: The environmental signal, the facial expression signal, and the physiological signal are preprocessed to obtain a plurality of feature values ​​for the environmental signal, the facial expression signal, and the physiological signal; The feature values ​​are normalized to map each feature value to the same range; and the probability value is determined based on the feature values ​​of the EEG signal, the environmental signal, the facial expression signal, and the physiological signal.

4. The recording method as described in claim 3, wherein the step of determining an emotion intensity value to characterize the emotion intensity of the user to be recorded includes: The initial multiple first weight values ​​of the electroencephalogram (EEG) signal, the environmental signal, the facial expression signal, and the physiological signal are obtained respectively. Based on the variation amplitude of the EEG signal, the environmental signal, the facial expression signal, and / or the physiological signal feature value, adjust each of the first weight values ​​to determine a plurality of second weight values ​​for the EEG signal, the environmental signal, the facial expression signal, and the physiological signal respectively: where is the first weight value of the i-th signal, are the second weight values ​​of the i-th signal, is the variation amplitude of the feature value of the i-th signal, and is the weight adjustment factor; and based on each of the second weight values ​​and the feature value of each of the signals, determine the emotion intensity value: where is the feature value of the i-th signal.

5. The recording method as claimed in claim 4, wherein the step of adjusting each of the first weight values ​​of the electroencephalogram (EEG) signals to determine each of the second weight values ​​of the EEG signals comprises: Based on the weight adjustment factor and the characteristic values ​​of the EEG signal, a weight adjustment value for the EEG signal is determined: where is the weight adjustment value of the EEG signal, is the weight adjustment factor, and is the characteristic value of the EEG signal; and based on the first weight value and the weight adjustment value, the second weight values ​​of the EEG signal are determined: where is the first weight value of the EEG signal, and is the second weight value of the EEG signal.

6. The recording method as described in claim 3 further includes the following steps: In response to the probability value being greater than or equal to the first threshold, adjusting the emotion intensity value according to the probability value: wherein, Here, is the adjusted emotional intensity value, is the original emotional intensity value, and is the probability value.

7. The recording method as described in claim 3, wherein after recording the event content of the real event, it further includes the following step: automatically generating an emotion fluctuation analysis report of the user to be recorded based on video and / or audio data during the period of the user's emotional fluctuation, wherein, The emotion fluctuation analysis report shall include at least the duration of the emotion fluctuation, the peak intensity, and the time of occurrence.

8. The recording method as described in claim 3, wherein after recording the event content of the real event, the method further includes the following steps: when the probability value is greater than or equal to the first threshold, or the emotion intensity value is less than a preset fourth threshold and its fluctuation duration is less than the third threshold, or the change amplitude of the signal characteristic values ​​of each of the EEG signals, the environmental signals, the facial expression signals and the physiological signals is lower than a preset fifth threshold, a second instruction to stop recording the user to be recorded is triggered.

9. A recording apparatus for implementing the method described in any one of claims 1 to 8, comprising: A signal acquisition module is used to acquire a plurality of multimodal signals of a user to be recorded, wherein the signal acquisition module includes at least an EEG sensor, an environmental sensor, a camera, and a smart wearable device, and the multimodal signals include at least EEG signals, environmental signals, facial expression signals, and physiological signals; a data processing module is used to determine, based on the multimodal signals, a probability value and an emotion intensity value for characterizing the user's emotions as originating from virtual events; and a recording module is used to selectively record the event content of real events that cause emotional fluctuations in the user to be recorded in response to the judgment of the data processing module.

10. A computer-readable storage medium storing computer instructions that, when executed by a processor, can implement a method for recording emotional fluctuation events as described in any one of claims 1 to 8.

Images