Speaker emotion perception method fusing multi-dimensional information

Abstract

The invention discloses a speaker emotion perception method fusing multi-dimensional information, and relates to the technical field of deep learning and human emotion perception. The method includes: inputting a video of a speaker, and extracting an image and voice of the speaker from the video; inputting the image and the voice of the speaker into a multi-dimensional feature extraction network, extracting a language text and a language emotion in the voice, and extracting facial expression features of the speaker from image information; utilizing a multi-dimensional feature coding algorithm for coding various feature results of the multi-dimensional feature extraction network, and mapping the multi-dimensional information to a shared coding space; fusing the features in the coding space from low dimension to high dimension by using a multi-dimensional feature fusion algorithm, and obtaining feature vectors of multi-dimensional information highly related to the emotion of the speaker in a high-dimensional feature space; and inputting the fused multi-dimensional information into an emotion perception network for prediction, and outputting emotion perception distribution of the speaker. By means of the method, the ambiguity can be effectively eliminated according to the multi-dimensional information, and the emotion perception distribution of the speaker can be accurately predicted.

Description

technical field [0001] The invention relates to the technical field of deep learning and human emotion perception, in particular to a speaker emotion perception method that integrates multi-dimensional information. Background technique [0002] The traditional deep learning algorithm only estimates the emotion of the language content information, and the language content itself is ambiguous, and needs to be combined with the intonation information when expressing the content, lacking the association and constraints between the language content and the voice emotional information, or only relying on Simple image information detection of faces for emotion estimation lacks adaptability to language content and voice emotions, and cannot be used in complex and changeable human-computer interaction scenarios in real situations, and its practical value is limited. [0003] Traditional emotion perception estimation methods based on deep learning can be divided into three parts: (1) ...

AI Technical Summary

Problems solved by technology

No matter which method is used, in complex and changeable interactive scenarios in reality, there are problems such as insufficient precision and inability to eliminate ambiguity.

There are also some methods that combine the semantic information of the language text and the audio signal to improve the effect of emotional perception to a certain extent, but the lack of image information of the speaker always limits the improvement effect

Images