Video description generation method based on deep learning and probabilistic graphical model

Abstract

The invention discloses a video description generation method based on deep learning and a probabilistic graphical model. The method systematically includes the steps that an existing image data set training rapid region object recognition convolution neural network model is utilized; an existing video data set training action recognition convolution neural network model is utilized; frame-extracting processing is carried out on a video, the rapid region object recognition convolution neural network model and the action recognition convolution neural network model are utilized for recognizing objects and action in the video, and the main content of the video is basically determined; by the utilization of a conditional random field, a subject-verb-object triad <objects, action and objects> with the maximum probability is found, the noise objects in the video are excluded, and a final description result is more accurate; the subject-verb-object triad is input into a long and short period memory network to input appropriate sentences which are namely description of the input video. The video is converted into the description, in this way, people can understand the content of the video more rapidly, and the retrieval speed of the video is increased ether.

Description

technical field [0001] The invention belongs to the technical field of video description generation, in particular to a video description generation method based on deep learning and a probability graph model. Background technique [0002] With the rapid development of the Internet, multimedia data such as text, voice, image and video have entered the era of big explosion. The popularity of smart devices equipped with rich sensors has promoted the development of user-generated content, and the threshold for generating data has become lower and lower, which has led to an exponential increase in the amount of data stored by Internet companies. Massive data provide basic conditions for scientific research and application. Emerging crowdsourcing methods provide data sets for model training, making data analysis enter a new stage. [0003] In an age of such huge data volumes, the speed at which information in streaming media can be absorbed becomes very important. Compared wit...

AI Technical Summary

Problems solved by technology

This method only uses the video data set for training, and the additional information used is insufficient, so it has certain limitations.

Images