Visual speech multi-mode collaborative analysis method based on emotion context and system

Abstract

The invention discloses a visual speech multi-mode collaborative analysis method based on an emotion context and a system. The method includes (S1) dynamically extracting and analyzing emotion context information based on a situation and an analysis object in a visual speech situation; (S2) extracting visual emotion characteristics of the analysis object in the visual situation and speech emotion characteristics of the analysis object in a speech situation in real time; (S3) carrying out structural sparse representation for the emotion context information, posture characteristics, expression characteristics and the speech emotion characteristics; and (S4) carrying out collaborative analysis and identification for the multi-mode emotion information by means of sentiment classification agents. Due to the facts that the emotion context information, the posture characteristics, the speech emotion characteristics and the expression characteristics include large amounts of emotion information, the emotion information is complementary, and the structural sparse representation and collaborative analysis through multiple sentiment agents are combined, emotion of a to-be-analyzed person is precisely analyzed on the condition that partial channel information is lost, and precision and robustness of emotion analysis in the natural interactive environment can be improved.

Description

technical field [0001] The present invention relates to the technical field of emotion recognition, in particular to a method and system for emotional context-based multimodal collaborative emotion analysis of visual and voice. Background technique [0002] With the development of multimedia technology, research on emotion analysis and recognition based on audio and video is of great significance for enhancing the intelligence and humanization of computers, developing new human-machine environments, and promoting the development of multimedia technology and signal processing and other related fields. Existing emotion recognition research still has many limitations. The research results mainly focus on single-channel emotion analysis. Existing emotion analysis research on multi-channel fusion also mainly focuses on two channels of expression and voice. The research objects are limited. It is within the range of a few performers in the laboratory, and does not consider the rea...

AI Technical Summary

Problems solved by technology

Existing emotion recognition research still has many limitations. The research results mainly focus on single-channel emotion analysis. Existing emotion analysis research on multi-channel fusion also mainly focuses on two channels of expression and voice. The research objects are limited. It is within the range of a few performers in the laboratory, and does not consider the real scene of the activity, ignoring the context of the analysis object, which leads to a significant decline in the recognition accuracy of the current emotion recognition method after it is separated from the experimental environment. The application effect is not satisfactory

Images