Method for segmenting and indexing scenes by combining captions and video image information

Abstract

The invention relates to a method for segmenting and indexing scenes by combining captions and video image information. The method is characterized in that: in the duration of each piece of caption, a video frame collection is used as a minimum unit of a scene cluster. The method comprises the steps of: after obtaining the minimum unit of the scene cluster, and extracting at least three or more discontinuous video frames to form a video key frame collection of the piece of caption; comparing the similarities of the key frames of a plurality of adjacent minimum units by using a bidirectional SIFT key point matching method and establishing an initial attribution relationship between the captions and the scenes by combining a caption related transition diagram; for the continuous minimum cluster units judged to be dissimilar, further judging whether the minimum cluster units can be merged by the relationship of the minimum cluster units and the corresponding captions; and according to the determined attribution relationships of the captions and the scenes, extracting the video scenes. For the segments of the extracted video scenes, the forward and reverse indexes, generated by the caption texts contained in the segments, are used as a foundation of indexing the video segments.

Description

technical field [0001] The present invention relates to the technical field of video indexing and searching, in particular, the present invention relates to a method for combining subtitles and video image information for scene segmentation and indexing. Background technique [0002] The advancement of mass storage device manufacturing technology, the improvement of network data transmission rate, and the continuous improvement of high-efficiency video compression technology have enabled the widespread dissemination and use of digital video, improving people's entertainment and cultural life. Finding interesting video clips in massive video databases has become a new problem. Videos can be organized into tree hierarchies containing scenes, shots, and frames. A frame is an image, which is the most basic physical unit in a video. A shot is a sequence of frames captured continuously by the same camera, which is the physical boundary of a sequence of video images. A scene is c...

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to overcome the unsatisfactory recall rate and precision rate of video scene extraction in the prior art, and the need to artificially mark the extracted video scene segments for video scene search, resulting in unobjective labeling results And the work is time-consuming and laborious, thus providing a method for joint subtitle and video image information for scene segmentation and indexing

Images