A Cross-Modal Sentiment Classification Method Based on Compact Bilinear Fusion of Image and Text

Abstract

The present invention provides a graphic-text cross-modal emotion classification method based on compact bilinear fusion, comprising the following six steps: (1) extraction of image feature representation; (2) extraction of text feature representation; (3) soft attention (4) Generation of image attention feature representation; (5) Multimodal compact bilinear fusion algorithm to fuse image attention feature representation and text feature representation; (6) Image-text emotion classification. The use of the soft attention map and the multimodal compact bilinear fusion algorithm in the method of the present invention can effectively improve the accuracy of emotion classification.

Description

technical field [0001] The invention relates to a cross-modal emotion classification method combining graphics and text, in particular to a cross-modal emotion classification method based on compact bilinear fusion of graphics and text. Background technique [0002] The object of cross-modal emotion classification research is to determine whether the sender is positive or negative based on the images sent by users and the corresponding text information on platforms such as Twitter, Facebook, Weibo, and e-commerce. manner. The application fields of graphic-text cross-modal sentiment classification mainly include: public opinion monitoring, user behavior analysis, product attribute evaluation, etc. With the advancement and development of science and technology, and the continuous improvement of software and hardware levels, many platforms can support users to send multimedia information such as voice, image, and video at the same time, and most of them choose to send both ima...

AI Technical Summary

Problems solved by technology

However, researchers only pay attention to the research of models, and cannot perform cross-modal sentiment classification of graphics and texts well.

Images