Video feature learning method based on video and text pair discriminant analysis

Abstract

The invention discloses a video feature learning method based on video and text description pair discriminant. A video-text pair is formed by the video and the text description matched with the video;a three-dimensional convolutional network is adopted to extract video features, a DistilBERT network is adopted to extract text description features, the video and the corresponding text descriptionhave similar semantic features through training, the text description automatically becomes a label of the corresponding video, and a deep learning network is trained and constructed and used for learning the video features. According to the method for performing video feature learning by using text description information as auxiliary information, efficient video feature representation can be learned while the labor cost is effectively reduced, and all data in a data set can be more effectively utilized to obtain video representation with higher discrimination capability by using a video feature learning method by using the video and the text description for discriminant.

Description

technical field [0001] The invention belongs to the technical field of computer software, relates to video representation technology, in particular to a video feature learning method based on video and text description pair discrimination. Background technique [0002] With the explosive growth of video data on the Internet, the demand for intelligent video analysis continues to rise, and the basis and key of video analysis is to obtain video features that can effectively describe the information contained in the video. Construct a variety of video analysis specific applications on the platform. Learning efficient video features using deep learning techniques has become a commonly adopted method, and these methods can be roughly divided into three categories. [0003] The first application is to use a 3D convolutional neural network trained on a human-annotated action recognition dataset to learn video features. In recent years, many manually labeled large-scale action rec...

AI Technical Summary

Problems solved by technology

[0005] The problem to be solved by the present invention is: most of the current learning methods of video features use manually marked action category labels as supervision information to train three-dimensional convolutional neural networks to perform action recognition tasks. There are also some methods that use the structure of the video itself to design a proxy task to train the network to learn video features. The video features learned by this type of method are poor and cannot effectively identify the video features contained in the video. information to extract

Images