A Multimodal Emotion Recognition Classification Method

Abstract

The present invention provides a multimodal emotion recognition and classification method, the method includes processing the video containing human faces to be detected and the video containing body movements corresponding to the same time, and converting them into image time series composed of image frames , extract the temporal features and spatial features in the image time series, based on the obtained multi-layer deep spatio-temporal features, perform multiple feature-level fusion on the features, and perform decision-level fusion on the classification results, so as to identify the video to be detected from multiple modalities The emotion type of the task, the method provided by the present invention makes full use of the effective information existing in each modality, and improves the recognition rate of emotion recognition.

Description

technical field [0001] The present invention relates to the technical field of computer processing, and more specifically, to a multimodal emotion recognition and classification method. Background technique [0002] Emotion recognition is an emerging research field interdisciplinary in computer science, cognitive science, psychology, brain science, neuroscience, etc. Its research purpose is to let computers learn to understand human emotional expressions, and ultimately enable them to recognize, The ability to understand emotions. Therefore, as a very challenging interdisciplinary subject, emotion recognition has become a research hotspot in the fields of pattern recognition, computer vision, big data mining and artificial intelligence at home and abroad, and has important research value and application prospects. [0003] In the existing emotion recognition technology, the research trend of emotion recognition presents two obvious characteristics. On the one hand, the data...

AI Technical Summary

Problems solved by technology

[0005] In order to solve the problem that the effective spatio-temporal features cannot be extracted from the video data for emotion recognition in the prior art, and whether the fusion in the early stage or the late stage is used in emotion recognition, similar fusion methods have the characteristics of model independence and are not sufficient. Using the effective information in each modality, there is a common problem of low fusion efficiency, and a multi-modal emotion recognition and classification method is provided.

Images