Video attribute representation learning method and method for automatically generating video text description

Abstract

The invention discloses a video attribute representation learning method, which comprises the following steps of collecting a batch of data for training and testing of an automatic description algorithm of video text, wherein the data demand that each video corresponds to a few of sentences of corresponding text description; defining all nouns, verbs and adjectives appearing in the text description content in a training set as attribute labeling information of corresponding videos, wherein each video in the training set corresponds to a plurality of attribute tags; and representing a segment of video sequence as a single image, thereby transforming a complicated and difficult video sequence multi-classification problem into a simple single-image multi-label classification problem. The invention further discloses a method for automatically generating video text description on the basis of the video attribute representation learning method. The video attribute representation learning method has the advantages that efficient attribute characteristic representation of extracted videos is provided, by adopting a fusion method disclosed by the invention, the intact method for automatically generating the text description capable of reflecting semantic information of video attributes can be obtained.

Description

technical field [0001] The invention relates to the field of computer vision, and more specifically to a method for automatically generating video text descriptions. Background technique [0002] The automatic generation of video text description refers to the automatic generation of a text description related to the video content through an algorithm given a video sequence. Due to the complexity of video content, traditional search model-based or language model-based algorithms have always been less effective. In recent years, with the development of deep learning technology, algorithms based on convolutional neural network (CNN) plus recurrent neural network (RNN) have achieved exciting results. The basic steps of this series of algorithms are as follows: (1) Extract the feature vector of the video through CNN (two-dimensional convolution or three-dimensional convolution), and then encode the video feature vector into the feature vector required by the language model thro...

AI Technical Summary

Problems solved by technology

[0005] The second technical problem to be solved by the present invention is to provide a video text description automatic generation method for how to integrate the extracted video semantic information with the existing video text description automatic generation codec model to improve the quality of the final generated text description

Images