A method and system for emotion recognition of short gif video based on text information

Abstract

The invention discloses a method and system for emotional recognition of GIF short videos fused with text information. The method first uses 3D convolutional neural network to extract sequence features in GIF short videos, and at the same time uses convolutional neural network to extract visual features of images in the sequence ; and then use the convolutional long-short-term memory recurrent neural network technology to decode the high-level semantic features after the fusion of the two, calculate the emotion classification probability distribution matrix, and perform interval mapping to obtain the emotional score of the video part. Then, the words containing emotional information are screened out from the words in the annotation text, and the emotional score of the text is calculated using the emotional scoring tool. Finally, the emotional score of the video and the emotional score of the text are added with different weights and the effectiveness is judged to classify the sentiment of the GIF short video. The invention can effectively pay attention to the emotional information of the image in the GIF video, and at the same time take into account the timing characteristics of the video stream, and integrate the text information and the video information, thereby improving the accuracy and robustness of the emotional classification of the GIF video.

Description

technical field [0001] The invention relates to a video emotion recognition method and system, in particular to a GIF short video emotion recognition method and system fused with text information, belonging to the technical field of computer vision video emotion recognition. Background technique [0002] Emotion recognition in video sequences is a research topic involving computer vision, pattern recognition and artificial intelligence. With the rapid development of Internet technology and the maturity of social media, more and more people use short videos to express their opinions and emotions. However, due to the existence of semantic gap and the difficulty of learning emotional information from video sequences in short video emotion recognition, it is very challenging to propose a method that can not only learn video sequence information but also accurately understand emotional semantics. [0003] Traditional emotion recognition work mainly focuses on text content that c...

AI Technical Summary

Problems solved by technology

However, the CNN used in the current relevant literature cannot describe the information in the time domain of GIF short videos. At the same time, the early fusion of video and text will increase the complexity of the model, and the fusion of video information and text information at the feature level cannot be judged. Importance of two kinds of information in final sentiment classification

Images