Segmentation method for space-time consistency of video sequence of parameter and depth information of known video camera

Abstract

The invention discloses a segmentation method for the space-time consistency of a video sequence of the parameter and depth information of a known video camera, comprising the following steps of: (1) segmenting videos by using a Mean-shift method; (2) counting Mean-shift segmentation boundaries according to the parameter and depth information of the video camera, and computing a probability boundary diagram of each frame; (3) segmenting the probability boundary diagram by utilizing a Watershed and an energy optimization method to obtain a more consistent segmentation result; (4) matching and connecting segmentation blocks among different frames to generate segmentation blocks in a space-time domain in regard to initialized segmentation obtained by utilizing the Watershed and the energy optimization method; (5) counting the probability of each pixel belonging to each space-time domain segmentation block by utilizing the parameters and the depth of the video camera, and iteratively optimizing frame by frame by using the energy optimization method to obtain a space-time consistent video segmentation result. The segmentation result can well keep object boundaries and the high space-time consistency of the segmentation blocks among multiple frames of the videos without flashing and skipping.

Description

technical field [0001] The invention relates to a time-space consistent segmentation method for video, in particular to a time-space consistent segmentation method for video sequences with known camera parameters and depth information. Background technique [0002] Video segmentation technology is one of the basic technologies in the field of computer vision, and it has important applications in many fields such as object recognition, image / video annotation, video editing and video coding. These applications usually require the results of video segmentation to have high spatio-temporal consistency. [0003] The difficulty of video segmentation technology lies in the high spatiotemporal consistency of the segmentation results. At present, the single image segmentation technology can better maintain the spatial domain segmentation boundary to a certain extent. For example, Mean-shift is a classic image segmentation method based on feature space clustering: D.Comaniciu, P.Meer...

AI Technical Summary

Problems solved by technology

The disadvantage of the independent processing method is the lack of accurate two-dimensional motion estimation, which affects the matching or transfer of the corresponding area; in addition, the complex area occlusion situation will also introduce many problems for area matching or transfer

The disadvantage of the joint processing method is that it cannot handle large-scale motion and severe occlusion; the huge amount of data also makes this type of method a serious memory overhead, and is not suitable for long video sequence processing

Images