Automatic video annotation method based on multi-modal private features

Abstract

The invention discloses an automatic video annotation method based on multi-modal private features, and the method comprises the steps: carrying out the preprocessing and manual annotation of a videofile, and filtering a manual annotation result; utilizing a generative adversarial network to extract common features among different modal features; stripping common features in the original featuresto obtain private features of different modes; integrating the extracted common features and modal private features to form new features of the video, and learning by using a multi-marking algorithmto obtain an automatic video marking classifier; sending the to-be-labeled video sample into a classifier to obtain a classification result, and realizing automatic labeling; and performing sampling inspection on the labeling result. By adopting the method, the classification model for automatic video annotation can be trained, the video features are integrated again by utilizing the private features of different modes of unknown annotated videos, the annotation task is automatically completed, and the manual annotation time and cost can be remarkably reduced.

Description

technical field

[0001] The invention relates to a video automatic labeling method, in particular to a video automatic labeling method suitable for video classification with multi-modal features and multi-label descriptions. Background technique

[0002] In recent years, various short video applications have emerged one after another, and users often use such applications for entertainment in scattered time. The emergence of short video applications makes the way for users to accept new things no longer limited to static text or pictures, and can be cleverly Taking advantage of time intervals, the number of such applications and short videos has shown explosive growth. But the ensuing question is how to ensure that users can search accurately, and how to ensure that users can make reasonable recommendations when they do not have a clear need to watch content. Using machine learning technology to automate search and recommendation is an effective method, and the basis of this...

Images