Emotion prediction method and system based on face and sound

Abstract

The invention discloses an emotion prediction method and system based on faces and sound, and the method comprises the steps: firstly collecting a face image and a voice recording sample, inputting the face image into a convolutional neural network, and extracting the spatial feature information in the image in a specific form; inputting the face image into a local binary convolutional network to extract texture feature information of the image, and fusing the spatial feature information and the texture feature information of the face image to output enhanced face features; then inputting voice recording into a preprocessing model to calculate the Mel spectrogram of each window, inputting the Mel spectrograms into a voice coding model to model the correlation of the short-term Mel spectrograms of each range, and outputting voice features through a voice segment embedding fusion device model; and finally, fusing the face features and the voice features, and inputting the fused features into an output model to predict emotion.

Description

technical field [0001] The invention relates to the field of computers, in particular to an emotion prediction method and system based on human face and voice. Background technique [0002] Emotion prediction is an important research area, which is widely used in various fields, including medical care, security and human-computer interaction, etc. Predicting emotions accurately and quickly is a daunting task due to the complexity of emotions expressed and manifested in different degrees or intensities. At present, emotion prediction mostly uses simple classification algorithms to identify and classify individual emotions, but the accuracy is not high and cannot meet the needs of practical applications. Contents of the invention [0003] The technical problem to be solved by the present invention is that the emotion prediction in the prior art usually uses a simple classification algorithm to identify and classify a single emotion, the accuracy is not high, and it cannot m...

AI Technical Summary

Problems solved by technology

[0003] The technical problem to be solved by the present invention is that the emotion prediction in the prior art usually uses a simple classification algorithm to identify and classify a single emotion, the accuracy is not high, and it cannot meet the needs of practical applications. The purpose is to provide an emotion prediction method based on human face and voice And the system can efficiently and accurately realize the emotion prediction task

Images