Attention fusion-based online short video multi-modal emotion recognition method

Abstract

The invention relates to the field of natural language processing, deep learning and multi-modal sentiment analysis, in particular to an attention fusion-based online short video multi-modal sentimentrecognition method, which comprises the following steps of: obtaining each single-modal feature in a short video; preprocessing the features by adopting a bidirectional GRU to obtain modal internal information; obtaining each advanced modal feature in combination with the interaction between the modal internal information and the modals; determining the contribution degree of each mode accordingto an attention mechanism to obtain a total feature vector, and inputting the total feature vector into a softmax function to obtain a bidirectional GRU multi-mode emotion recognition model based on attention fusion; training the model, and inputting a to-be-recognized short video into the trained model to obtain an emotion recognition result; according to the multi-modal emotion recognition method and system, all single-modal features are well fused, emotion information expressed in the video is effectively mined, and therefore the accuracy and efficiency of multi-modal emotion recognition are improved.

Description

technical field [0001] The invention relates to the fields of natural language processing, deep learning, and multimodal emotion analysis, and in particular to an online short video multimodal emotion recognition method based on attention fusion. Background technique [0002] With the widespread popularization of the Internet, the scale of mobile Internet users continues to expand, and more and more people communicate through the Internet. Therefore, a large number of users' valuable comment information on people, events, products, etc. have been generated on the Internet. These comment information It expresses people's emotional color and emotional tendency. However, with the advancement of communication technology and the rapid rise of emerging social media (such as Douyin, Miaopai, Kuaishou, etc.), online short videos have attracted more and more attention from people, and people are used to expressing their emotions by taking short videos or opinion. With the increase ...

AI Technical Summary

Problems solved by technology

Although this method fully considers the differences in the characteristics of each mode, it is difficult to learn the interrelationships between modes.

Images