Speech emotion recognition method based on multi-scale deep convolution recurrent neural network

Abstract

The invention discloses a speech emotion recognition method based on a multi-scale deep convolution recurrent neural network. The method comprises the steps that (1), three-channel speech spectrum segments are generated; (2), speech spectrum segment features under different scales are extracted by adopting the convolution neural network (CNN); (3), time modeling of a speech spectrum segment sequence under different scales is achieved by adopting a long short-term memory (LSTM), and emotion recognition results of a whole sentence of speech is output; (4), fusions of recognition results obtainedby CNN+LSTM under different scales are achieved by adopting a score level fusion method, and the final speech emotion recognition result is output. By means of the method, natural speech emotion recognition performance under actual environments can be effectively improved, and the method can be applied to the fields of artificial intelligence, robot technologies, natural human-computer interaction technologies and the like.

Description

technical field [0001] The invention relates to the fields of speech signal processing and pattern recognition, in particular to a speech emotion recognition method based on a multi-scale deep convolutional cyclic neural network. Background technique [0002] Human language not only contains rich text information, but also carries audio information that contains people's emotional expressions, such as changes in the pitch, strength, and cadence of voice. How to let the computer automatically recognize the emotional state of the speaker from the voice signal, that is, the so-called "speech emotion recognition" research, has become a hot research topic in the fields of artificial intelligence, pattern recognition, and emotional computing. The research aims to enable the computer to acquire, recognize and respond to the user's emotional information by analyzing the speaker's voice signal, so as to achieve a more harmonious and natural interaction between the user and the comput...

AI Technical Summary

Problems solved by technology

However, the existing speech emotion recognition methods based on deep learning technology have ignored the different characteristics of different lengths of speech spectrum fragment information for the different discrimination of different emotion types (see literature: Mao Q, et al. Learning salient features for speech emotion recognition using convolutional neural networks. IEEE Transactions on Multimedia, 2014, 16(8): 2203-2213.)

Images