Image emotion recognition method and system based on multi-modal data

Abstract

The invention relates to an image emotion recognition method and system based on multi-modal data. The method comprises the steps of obtaining training sample data; obtaining a trained recurrent neural network and a trained convolutional neural network according to the training sample data; acquiring to-be-identified data; inputting to-be-recognized text data into the trained recurrent neural network to obtain text features; inputting a to-be-recognized image into the trained convolutional neural network to obtain aesthetic features, emotion features and shared features; fusing the aesthetic feature, the shared feature and the text feature by adopting a TFN method to obtain a first to-be-identified fused feature; fusing the emotion feature, the image feature and the text feature by adopting a TFN method to obtain a second to-be-recognized fusion feature; and determining the emotion of the to-be-identified data according to the to-be-identified fusion feature. By means of the method, the emotion recognition accuracy is improved.

Description

technical field [0001] The invention relates to the technical field of image processing, in particular to an image emotion recognition method and system based on multimodal data. Background technique [0002] Sentiment analysis of multimedia data has always been a challenging task. Many scholars and companies at home and abroad have carried out research on sentiment analysis of various modal data. But for a long time, researchers have mainly focused on the sentiment analysis algorithm of a single modality, and paid less attention to the joint analysis of multiple modality data. [0003] The traditional sentiment analysis method extracts the texture of the picture, clusters various colors, and constructs the correlation between color, shape, texture and emotion. However, the emotion of an image is affected by many aspects, and the factors considered by traditional emotion analysis methods are not comprehensive and accurate enough, resulting in poor accuracy of emotion recogn...

AI Technical Summary

Problems solved by technology

However, the emotion of an image is affected by many aspects, and the factors considered by traditional emotion analysis methods are not comprehensive and accurate enough, resulting in poor accuracy of emotion recognition.

Images