Global and local feature embedding and image-text fusion sentiment analysis method and system

Abstract

The invention discloses a global and local feature embedding and image-text fusion sentiment analysis method and system. The method comprises the following steps: firstly, extracting global features of an image by using a convolutional neural network, training a target detection model by using transfer learning, detecting and positioning a local region carrying emotion in the image, extracting local region features, and embedding the local region features into the global features to jointly train an image emotion classification model to obtain an emotion polarity probability of the image; expressing the text as a word vector containing rich semantic information, and inputting the word vector into a bidirectional LSTM capable of extracting text semantic context features to perform sentimentclassification to obtain the polarity probability of sentiment of the text; and finally, according to a later fusion formula, obtaining an emotion polarity probability after image and text fusion, and performing image-text emotion analysis. The method can effectively pay attention to the emotion information of the image-text, and improves the accuracy and robustness of image-text emotion classification through the extraction of global and local emotion features of the image and the fusion of text information.

Description

Technical field [0001] The present invention relates to an emotion analysis method and system for global and local feature embedding and image and text fusion, in particular to the emotion analysis of social media images and text, and belongs to the technical field of emotion recognition intersected by computer vision image and natural language processing. Background technique [0002] The sentiment analysis of images and text is a research topic involving computer vision, pattern recognition, and natural language processing. With the continuous development of the Internet and the maturity of social media, more and more Internet users like to share their experiences on social platforms such as Weibo and WeChat and participate in discussions on various events and topics. Social networking sites have become Internet users An important platform for opinions and feelings. By analyzing the opinions and pictures shared by users on social media, mining users’ emotional tendencies can b...

AI Technical Summary

Problems solved by technology

This method combines the global and local features of the image to jointly mine the emotion of the image, further improves the accuracy of image emotion recognition, and integrates the emotional information of the text to solve the pr

Images