Video highlight detection method and device based on graph neural network

Abstract

The invention relates to the technical field of video information, in particular to a video highlight detection method and device based on a graph neural network. In order to solve the problem of lowdetection precision of video highlights in the prior art, the invention provides the method, which comprises the steps of obtaining image feature information of each frame of image in a to-be-detectedvideo through a preset image feature extraction model based on the to-be-detected video obtained in advance; constructing a space diagram corresponding to each frame of image based on the image feature information of each frame of image; according to the space diagram corresponding to each frame of image, obtaining semantic features of the object in each frame of image through a preset semantic feature extraction model, and constructing a time sequence diagram corresponding to each frame of image according to the semantic features of the object in each frame of image; and according to the time sequence diagram corresponding to each frame of image, obtaining a user interest score of each frame of image in the to-be-detected video through a preset video clip detection model. The detection accuracy of the video highlight is improved.

Description

technical field [0001] The invention relates to the technical field of video information, in particular to a method and device for detecting video highlights based on a graph neural network. Background technique [0002] With the popularity of wearable devices such as camcorders and smart glasses, more and more people record their lives through video, and video highlight detection is becoming increasingly important. [0003] Most of the existing video highlight detection methods extract the overall features of the video, and do not take into account the differences in spatio-temporal local features. Due to the complexity of video content, such mixed features will affect the final highlight detection performance. Existing models are mainly divided into three types, namely, ranking models based on hidden variables, models based on autoencoders and models based on convolutional neural networks. The model based on latent variables solves the problem of a large amount of noise ...

AI Technical Summary

Problems solved by technology

[0005] In order to solve the above-mentioned problems in the prior art, that is, in order to solve the problem of low detection accuracy of video highlights in the prior art, the first aspect of the present invention provides a method for detecting video highlights based on a graph neural network, the method include:

Images