Multimodal emotion recognition method and system integrating attention mechanism and dmcca

Abstract

The invention discloses a multimodal emotion recognition method and system integrating attention mechanism and discriminating multiple set canonical correlation analysis (DMCCA). The method includes: extracting EEG signal features, peripheral physiological signal features and expression features respectively from preprocessed EEG signals, peripheral physiological signals and facial expression videos; using an attention mechanism to extract discriminative EEG emotional features respectively , peripheral physiological emotion feature, expression emotion feature; use DMCCA method on EEG emotion feature, peripheral physiological emotion feature and expression emotion feature to obtain EEG-peripheral physiological-expression multimodal emotion feature; use classifier to classify multimodal emotion Features are classified and identified. The invention adopts the attention mechanism to selectively focus on the more emotionally discriminating features in each modal, and makes full use of the correlation and complementarity between the emotional features of different modalities in combination with DMCCA, which can effectively improve the accuracy of emotion recognition. and robustness.

Description

technical field [0001] The present invention relates to the technical field of emotion recognition and artificial intelligence, and in particular, to a multimodal emotion recognition method and system integrating attention mechanism and discriminative multiple set canonical correlation analysis (DMCCA). Background technique [0002] Human emotion is the psychological and physiological state that accompanies the process of human consciousness and plays an important role in interpersonal communication. With the continuous progress of artificial intelligence and other technologies, obtaining a more intelligent and humanized human-computer interaction (Human–Computer Interactions, HCIs) experience has attracted more and more attention. People have higher and higher requirements for machine intelligence, expecting machines to have the ability to perceive, understand and even express emotions, realize human-computer interaction, and better serve human beings. As a branch of affec...

AI Technical Summary

Problems solved by technology

[0005] Purpose of the invention: for the shortcomings of low single-modal emotion recognition accuracy, poor robustness and existing multi-modal emotional feature fusion methods, the purpose of the invention is to provide a fusion attention mechanism and identify multiple sets of canonical correlation analysis ( DMCCA) multi-modal emotion recognition method and system, by introducing an attention mechanism to selectively focus on the discriminating emotional features in each modality, and combined with DMCCA to make full use of the correlation and complementarity between different modal emotional features can effectively improve the accuracy and robustness of multi-modal emotion recognition

Images