Video emotion classification method and system fusing electroencephalogram and stimulation source information

Abstract

The invention discloses a video emotion classification method and system fusing electroencephalogram and stimulation source information, and the method comprises the steps: constructing a stimulation source-electroencephalogram signal data set: enabling a subject to watch a video clip, collecting the electroencephalogram signals of the subject when the subject watches a video through an electroencephalogram scanner, and constructing the stimulation source-electroencephalogram signal data set; and constructing a multi-modal feature fusion model: for the training data set, respectively extracting video features and electroencephalogram signal features, and generating a fusion vector by adopting a multi-modal information fusion method based on an attention mechanism. Training a fusion vector classification model: taking the fusion vector as the input of a neural network full connection layer for prediction; updating the weight of the neural network according to the difference between the prediction result and the real label, and training the neural network. Classifying by using the model; collecting an electroencephalogram signal when a subject watches a to-be-classified video; extracting and fusing video features and electroencephalogram signal features; and inputting the fusion vector into the trained neural network to obtain a classification result.

Description

technical field [0001] The invention relates to the field of multi-modal fusion video emotion classification, in particular to a video emotion classification method and system for fusing EEG and stimulus source information. Background technique [0002] Video emotion classification is a hot spot in the research direction of computer vision, and it also has broad application value. In the video recommendation system, by calculating the emotion of the user watching the video and obtaining their emotional preference, the video that is more in line with their preference can be recommended. In public opinion events, obtaining videos under specific hot topics, calculating the emotions of the videos for guidance, and establishing correct public opinion orientation is conducive to creating a harmonious and stable cyberspace environment. In addition, video sentiment classification is also of great significance in video classification and advertisement placement. [0003] Therefore,...

AI Technical Summary

Problems solved by technology

Existing technology 2 only pays attention to the emotion of the video barrage, but does not pay attention to the information of the video itself. At the same time, it cannot use this method to classify video clips without barrage

Existing technology 3 only pays attention to the EEG signal generated by the user watching the video, but does not pay attention to the information of the video itself, resulting in unsatisfactory classification effect

Images