Automatic video annotation method based on multi-modal private features

An automatic labeling, multi-modal technology, applied in video data retrieval, neural learning methods, video data clustering/classification, etc., to reduce the time and cost of manual labeling

Active Publication Date: 2019-10-25
SOUTHEAST UNIV
View PDF2 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

But the ensuing question is how to ensure that users can search accurately, and how to ensure...

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Automatic video annotation method based on multi-modal private features
  • Automatic video annotation method based on multi-modal private features
  • Automatic video annotation method based on multi-modal private features

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0028] The present invention will be described in further detail below in conjunction with accompanying drawing, as figure 1 As shown, the video module and the video tagging module store the original video data and all tagging sets. For the original video, the work of feature extraction needs to be completed first. A video can be described from different angles, such as the text description of the video title, the title image expressing the main content of the video, the video frame describing the detailed content of the video, and the audio describing the video expression. to multimodal video features. For video tagging, it is first necessary to select some video samples for manual tagging. In order to prevent taggers with different expressive abilities from using similar but not identical tags for tagging, all tags should come from the tag set. Afterwards, in order to ensure that the number of videos contained in different tags is relatively balanced, it is necessary to fi...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses an automatic video annotation method based on multi-modal private features, and the method comprises the steps: carrying out the preprocessing and manual annotation of a videofile, and filtering a manual annotation result; utilizing a generative adversarial network to extract common features among different modal features; stripping common features in the original featuresto obtain private features of different modes; integrating the extracted common features and modal private features to form new features of the video, and learning by using a multi-marking algorithmto obtain an automatic video marking classifier; sending the to-be-labeled video sample into a classifier to obtain a classification result, and realizing automatic labeling; and performing sampling inspection on the labeling result. By adopting the method, the classification model for automatic video annotation can be trained, the video features are integrated again by utilizing the private features of different modes of unknown annotated videos, the annotation task is automatically completed, and the manual annotation time and cost can be remarkably reduced.

Description

technical field [0001] The invention relates to a video automatic labeling method, in particular to a video automatic labeling method suitable for video classification with multi-modal features and multi-label descriptions. Background technique [0002] In recent years, various short video applications have emerged one after another, and users often use such applications for entertainment in scattered time. The emergence of short video applications makes the way for users to accept new things no longer limited to static text or pictures, and can be cleverly Taking advantage of time intervals, the number of such applications and short videos has shown explosive growth. But the ensuing question is how to ensure that users can search accurately, and how to ensure that users can make reasonable recommendations when they do not have a clear need to watch content. Using machine learning technology to automate search and recommendation is an effective method, and the basis of this...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/75G06K9/46G06K9/62G06N3/04G06N3/08
CPCG06F16/75G06N3/08G06V10/40G06N3/045G06F18/241
Inventor 张敏灵吴璇
Owner SOUTHEAST UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap