Cross-media data emotion recognition method based on facial expressions and electroencephalogram signals

Abstract

The invention discloses a cross-media data emotion recognition method based on facial expressions and electroencephalogram signals, and the method comprises the steps: obtaining original facial expression video data and original electroencephalogram data, and constructing a data set based on the original facial expression video data and the original electroencephalogram data; constructing a multi-modal neural network structure; and based on the data set and the multi-modal neural network structure, obtaining an emotion recognition result, and completing cross-media data emotion recognition. According to the method, the feature information of the facial expressions and the electroencephalogram signals is effectively fused and applied to multi-modal emotion recognition, the data of the two modals support and complement each other, and the objectivity and accuracy of emotion recognition can be effectively improved.

Description

technical field [0001] The invention belongs to the technical field of emotion recognition, and in particular relates to a cross-media data emotion recognition method based on human facial expressions and electroencephalogram signals. Background technique [0002] Affective computing is an interdisciplinary research field that integrates research directions such as artificial intelligence, computer vision, brain-computer fusion, and signal transmission and processing. It aims to explore and develop theories that can recognize, explain, process, and simulate human emotions. methods and systems. Affective computing has three research directions: expression, decision-making and recognition, among which emotion recognition has received more extensive attention at present. With the rapid development of artificial intelligence and deep learning technology, emotion recognition technologies based on convolutional neural network (CNN) have gradually emerged, most of which are single...

AI Technical Summary

Problems solved by technology

[0006] The accuracy of some existing emotion recognition methods is roughly in the range of 50% to 75%, and the reliability is poor in the process of practical application. Moreover, most of the existing emotion recognition methods are based on public facial expression images , EEG signals, or other physiological electrical signal data sets (such as SEED, DEAP, FER2013, etc.), although such data sets are relatively complete, due to their complete disclosure, criminals can easily obtain these data sets and Design a neural network model with a backdoor according to the characteristics of the data set, or use the confrontation network (GAN) to generate fake data to deceive the neural network model trained based on this type of data set, so that the accuracy and security of its application Greatly reduced

Images