Video content positioning method based on feature fusion and cascade learning

Abstract

The invention discloses a video content positioning method based on feature fusion. The method comprises a video feature extraction process, a feature fusion process and a video content positioning process. The feature fusion comprises pre-fusion and post-fusion; the pre-fusion comprises steps of cascading low-level features of each dimension of the video, and fusing the low-level features througha neural network shown in the specification; and the post-fusion comprises steps of splicing the outputs of the neural network shown in the specification, and fusing the high-level semantic featuresof each dimension of the video through the neural network CF (.). According to the method and the device, the problem of loss of complementary high-level semantic information contained in video imagesand sounds can be solved, and accurate positioning of video contents is realized.

Description

technical field [0001] The invention belongs to the technical field of machine vision and deep learning, and relates to a video content positioning technology, in particular to a video content positioning method using a cascaded neural network based on front and back fusion of features. Background technique [0002] Various videos contain a large number of events and actions, and the core content of the video is in these events. However, most videos are redundant and long-running, and viewers need to find useful information in a large amount of redundant information, so a technology that can automatically locate the required content segments is urgently needed. This kind of video content positioning is very helpful for subsequent content analysis and classification, and has great application space in the fields of security, education, and film and television. [0003] In the prior art about video content location methods, the neural network method is adopted, but the inform...

AI Technical Summary

Problems solved by technology

[0005] In the existing video content localization method, the feature extraction module of the neural network simply concatenates the feature vector of the video image and the feature vector of the video sound as the feature vector of the video, which will lose the complementary high-level semantics contained in the image and sound features. information, greatly reducing the accuracy of video content positioning, making it difficult to achieve precise positioning of video content

Images