A video emotion recognition method integrating facial expression recognition and voice emotion recognition

Abstract

The invention relates to a video emotion recognition method integrating facial expression recognition and voice emotion recognition. The invention relates to processing of a recording medium for identifying graphics. The method is a decision-level-based two-process progressive audio-visual emotion recognition method, the method separates face expression recognition from voice emotion recognition in a video, the two-process progressive emotion recognition method is adopted, and the technology of voice emotion recognition is carried out on the basis of face expression recognition by calculatingthe conditional probability; the method comprises the steps that A, facial image expression recognition serves as first-time classification recognition; B, voice emotion recognition serves as second-time classification recognition; and C, fusion of facial expression recognition and speech emotion recognition. According to the method, the defects that in the prior art, the internal relation betweenface features and voice features is ignored in human emotion recognition, the recognition speed of video emotion recognition is low, and the recognition rate is low are overcome.

Description

technical field [0001] The technical solution of the present invention relates to the processing of the record carrier used for recognizing graphics, in particular to a video emotion recognition method that integrates facial expression recognition and voice emotion recognition. Background technique [0002] With the rapid development of artificial intelligence and computer vision technology, human-computer interaction technology is changing with each passing day. Human emotion recognition technology using computers has received extensive attention. How to make computers recognize human emotions more quickly and accurately has become the current topic in the field of machine vision. Research hotspots. [0003] Human beings express their emotions in a variety of ways, mainly including facial expressions, voice emotions, upper body postures, and language texts. Among them, facial expressions and voice emotions are the two most typical ways of expressing emotions. Since the te...

AI Technical Summary

Problems solved by technology

The shortcomings of the existing decision-level fusion methods mainly include two points. First, the proportional scoring mechanism and weight allocation strategy lack unified and authoritative standards. Different researchers often use various proportional scoring mechanisms and different weight allocation strategies in the same research. Different recognition results were obtained in the project; second: the decision-level fusion method focuses on the fusion of face recognition and speech recognition results, ignoring the internal relationship between face features and speech features

[0007] CN106529504A discloses a dual-mode video emotion recognition method with composite spatio-temporal features, which expands the existing volume local binary mode algorithm into a spatio-temporal ternary mode, and obtains the spatio-temporal local ternary mode moment texture features of facial expression and upper body posture, Further integrate the three-dimensional gradient direction histogram feature to enhance the description of emotional video, and combine the two features into a composite spatio-temporal feature. This method will affect its algorithm when the upper body posture of the person in the video changes rapidly or the upper body posture picture is missing. Therefore, the dual-modal video emotion recognition method combined with facial expressio

Images