Shot Boundary Detection Method Based on Convolutional Neural Network

Abstract

The invention discloses a lens boundary detection method based on a convolution neural network, and the method comprises the steps: segmenting a video; calculating the local threshold values of each segment; removing the segment with the in-segment brightness distance being less than the corresponding threshold value, and carrying out double binarization of the segment with the in-segment brightness distance being greater than the corresponding threshold value; eliminating the segment which does not comprise the boundary of a lens according to a candidate segment and the in-segment brightness distance relation of small segments obtained through binarization, classifying the measurement result, and obtaining an abrupt change boundary candidate segment and a gradual change boundary candidate segment; extracting the features of each frame in the segments through the convolution neural network, and measuring the interframe similarity through employing the cosine distance between the features; determining whether the abrupt change boundary candidate segment has an abrupt change boundary or not and the position of the abrupt change boundary according to the relation between the interframe similarity of two continuous frames and the interframe similarity of the head and end frames of the candidate segment; calculating the absolute distance for the gradual change boundary candidate segment, drawing a curve, determining whether the gradual change boundary candidate segment is a gradual change boundary or not, and carrying out the integration of the adjacent gradual change boundaries. The method can accurately detect the abrupt change lens boundary and the gradual change lens boundary.

Description

technical field [0001] The invention relates to a shot boundary detection technology that can be used in video analysis and retrieval, in particular to a shot boundary detection method based on a convolutional neural network. Background technique [0002] Video shot boundary detection refers to detecting the position of the video shot boundary. A shot is defined as a sequence of video frames captured by a single camera without interference. There are two types of shot boundaries: one is a sudden shot boundary, which exists between two consecutive frames and consists of the last frame of the previous shot and the first frame of the next shot; the other is a gradual shot Boundary, the border of a progressive shot generally has more than two frames, it usually consists of some inter-frame related frames, and there will be a gentle transition from the last shot to the next shot. [0003] Video shot boundary detection can help analyze the content and structure of videos, and ca...

AI Technical Summary

Problems solved by technology

In the article "Fast video shot boundary detection framework employing pre-processing techniques" published by IET Image Process in 2009, Y.Li, Z.Lu, and X.Niu et al proposed a shot boundary candidate segment detection based on pixel brightness distance The preprocessing and gradient boundary detection method of triangle pattern matching on the brightness distance between frames greatly shortened the time of video boundary detection at that time, but it has two disadvantages: one is that the accuracy rate is not high, and the other is that it cannot meet the requirements. Requirements for real-time processing

Tong et al. considered the content information of the frame in the article "CNN-based shot boundary detection and video annotation" published by IEEE International Symposium on Broadband MultimediaSystems and Broadcasting in 2015, and used the content information to help detect the shot boundary. However, when dealing with gradient boundaries with similar backgrounds When , due to the limitation of the training database, the extracted content information is almost the same, and the detection effect will be relatively poor at this time.

Images