Overall emotion recognition method combining image and speech

Abstract

The present invention discloses an overall emotion recognition method and system combining an image and speech. The process of recognition comprises: after acquiring a corresponding speech and video signal from an input video, an information acquisition apparatus transmits the corresponding speech and video signal to corresponding emotion classification modules respectively, and after classification, an integrated learning trainer allocates weights, and after weighting, a recognition result is output to complete a recognition process. The system comprises an information acquisition apparatus, an emotion classifier and an integrated processor. The information acquisition apparatus comprises a video acquisition device and an audio acquisition device; the emotion classifier comprises an expression emotion classification module for performing emotion classification on acquired video information and a speech emotion classification module for performing emotion classification on acquired audio information; and the integrated processor comprises a weighting module and an integrated learning trainer. The method and system provided by the present invention have the advantages of higher emotion classification reliability, flexible adjustment on confidence parameters and high precision; and through bi-directional recognition of expression and speech, the human emotion recognition process is simulated to a large extent.

Description

technical field [0001] The invention belongs to the interdisciplinary technical fields of computer technology, information technology and data mining, and relates to a comprehensive emotion recognition method and system combining image and voice. In human-computer interaction, two aspects of the face image and voice of the same person are mainly used Emotion recognition with weighted assignment. Background technique [0002] Facial emotion recognition refers to the use of computers to extract and analyze human facial expression information, classify and understand according to the current public understanding and way of thinking, and integrate the prior knowledge of human emotional information to make the computer independently associate, think and reason. Finally, analyze human emotions from face information. Since facial expression recognition has broad application prospects, it has gradually become one of the research hotspots in the fields of human-computer interaction,...

AI Technical Summary

Problems solved by technology

[0005] At present, the existing human emotion recognition is only a single face image emotion recognition or a single voice emotion recognition. Using image emotion recognition under the background of dim light and blurred images, or using speech recognition under the background of environmental noise will make human Emotion recognition has a large error, and even misjudged it as other emotion categories

A single aspect of human emotion recognition technology has encountered a bottleneck, and the accuracy rate is difficult to improve

In addition, in human-computer interaction, the existing emotion recognition calculation is still huge, resulting in delay, which will cause data errors, which is very unfriendly in human-computer interaction

Images