Multi-modal emotion classification method based on text, voice and video fusion

Abstract

The invention discloses a multi-modal emotion classification method based on text, voice and video fusion, and the method comprises the steps of obtaining the multi-modal data, preprocessing the multi-modal data, and dividing the multi-modal data into a training set and a test set; constructing an end-to-end multi-modal emotion classification model based on tensor fusion, and training the model by using the training set; and carrying out the preprocessing operation in the step 1 on the test set, and carrying out the sentiment classification by using the tensor fusion sentiment classification model obtained in the step 2. According to the present invention, the fuzzy deep emotion information can be better captured through the multi-modal emotion classification model.

Description

technical field [0001] The invention belongs to natural language processing technology, specifically a multimodal emotion classification method based on fusion of text, voice and video. Background technique [0002] At present, relevant social media sites are producing a large amount of video data with rich emotional information every day, resulting in a large number of multimodal opinion mining and sentiment analysis technologies for text, voice, and video. This technology is not only a natural Academic frontier issues and hot research issues in the field of language processing and sentiment analysis are also important issues that need to be solved urgently in the field of application. They have immeasurable application value and social significance, but also pose great challenges. [0003] Before the rise of machine learning methods, sentiment analysis was mainly done manually, which was costly and unstable. Traditional machine learning and traditional multimodal methods ...

AI Technical Summary

Problems solved by technology

[0003] Before the rise of machine learning methods, sentiment analysis was mainly done manually, with high cost and unstable accuracy

Traditional machine learning and traditional multimodal methods mainly rely on the idea of ​​feature engineering, using artificially extracted features on the voice and video sides. However, due to the ambiguity of emotional expression, artificially extracted features are often difficult to extract deep emotional expressions. , there is still a lot of room for improvement in the accuracy of emotion recognition

Images