Speech emotion recognition method integrated with semantics

Abstract

The invention discloses a semantic-fused speech emotion recognition method, and relates to the technical field of speech emotion recognition. The method comprises the following steps: a computer inputs a video into a model based on a hierarchical attention mechanism to learn facial emotion significant features; the computer inputs the audio signal into a convolutional network guided by a frequency domain attention mechanism to learn significant frequency domain emotion feature information; the computer inputs the text word vectors into a semantic emotion feature extraction network to learn significant utterance text emotion features; and the computer fuses the audio emotion features and the semantic emotion features by using a multi-modal attention mechanism. According to the method, through a CNN neural network model framework An-Net fusing semantics and voice, the emotion in the speech and voice of a specific person can be accurately recognized in a complex scene, and a multi-mode emotion recognition model is adopted under the guidance of a hierarchical attention mechanism.

Description

technical field [0001] The invention belongs to the technical field of speech emotion recognition, in particular to a speech emotion recognition method incorporating semantics. Background technique [0002] Speech emotion recognition is one of the research hotspots of new human-computer interaction technology, and has a wide range of application prospects in artificial intelligence. Speech emotion recognition includes the establishment of emotional speech database, speech emotion feature extraction and speech emotion recognition classifier, etc. Recognition is to study the formation and changes of the speaker's emotional state from the perspective of speech signals, so as to make the interaction between computers and humans more intelligent. Features, sound quality features, and fusion features of the above features. [0003] The existing speech emotion recognition methods are difficult to accurately recognize the emotion and emotion of speech in complex scenes, and for a s...

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to provide a speech emotion recognition method that integrates semantics, which solves the problem that the existing speech emotion recognition method is difficult to accurately recognize the emotion and emotion of speech in complex scenes, and for a single frame with serious mixed noise, the processing effect is relatively low. Poor, technical problems that make it difficult to obtain satisfactory results

Images