Method and equipment for generating video content description information

Abstract

The invention relates to a method and equipment for generating video content description information. The method comprises the following steps: acquiring an image sequence of a target video; dividing the image sequence into a plurality of sub-image sequences; any two continuous subimage sequences in the plurality of subimage sequences corresponding to different scenes; for each subimage sequence in the plurality of subimage sequences, detecting the first frame image of the current subimage sequence according to a trained first detection model to obtain corresponding static scene description information; detecting images except the first frame image in the current sub-image sequence according to a trained second detection model to obtain corresponding dynamic event description information; and determining content description information corresponding to the current sub-image sequence according to the static scene description information and the dynamic event description information. Therefore, the video understanding difficulty can be reduced, redundant information extraction can be reduced, and the calculation efficiency can be improved.

Description

technical field [0001] The present application relates to the technical field of computer vision, in particular to a method and device for generating video content description information. Background technique [0002] In the context of the era of steady development of the Internet and big data, the demand for multimedia information has shown explosive growth, and traditional information processing technology has been unable to cope with the needs of multimedia data in labeling, description and other tasks. Describing videos, images, etc. in natural language is very simple for humans, but it is a difficult task for machines. [0003] At present, there have been many studies on the use of convolutional neural networks to process two-dimensional image data, but the processing methods for video data are still in the stage to be improved. [0004] In the existing video understanding method, based on the video data frame-level feature sequence, the global part-of-speech sequence...

AI Technical Summary

Problems solved by technology

[0002] In the context of the stable development of the Internet and big data, the demand for multimedia information has shown explosive growth, and traditional information processing technology has been unable to meet the needs of multimedia data in labeling, description and other tasks

Describing videos, images, etc. in natural language is easy for humans but a difficult task for machines

[0003] At present, there have been many studies on the use of convolutional neural networks to process two-dimensional image data, but the processing methods for video data are still in the stage to be improved.

[0004] In the existing video understanding method, based on the video data frame-level feature sequence, the global part-of-speech sequence feature corresponding to natural language is extracted, and then an accurate natural language description is generated, but this method does not exclude repeated information between consecutive frames in the same scene. High redundancy; another example is that in the existing technology, the video image sequence is screened for key frames, and then the screened key frames are sent to the video frame description network to generate description text, but this method does not consider the The correlation between continuous video frames and the information differences between scenes are not suitable for understanding videos with scene changes, such as non-fixed camera videos, and videos that have been edited and spliced ​​from multiple scenes such as film and television works, etc.

Images