Method for performing multi-body depth recovery and segmentation on video

Abstract

The invention discloses a method for performing multi-body depth recovery and segmentation on a video, which comprises the following steps of: (1) performing energy minimization on the video by an iteration method to obtain an initial label of each frame of the video, wherein the initial label consists of the depth of pixels and segmentation information; (2) after performing image segmentation on each frame, optimizing the initial label of each frame of image by a multi-body plane fitting method to obtain the optimized labels of all segmentation blocks in each frame of image; (3) selecting a group of visible frames and a group of invisible frames from adjacent frames for each pixel on each frame by using the optimized labels; and (4) performing the energy minimization on each frame of the video by the iteration method to obtain the iterated labels of each frame of the video, and expanding the progression of the depth of the iterated labels further by using a hierarchical confidence propagation algorithm. By the method, the depth recovery and segmentation can be performed on videos in which multiple rigid objects move.

Description

technical field [0001] The invention relates to a method for depth restoration and segmentation, which performs depth restoration and segmentation on videos with multiple rigid object movements. Background technique [0002] Depth-based 3D restoration and image (or video) segmentation have always been fundamental problems in computer vision, because the calculated depth images and segmented images can be used separately or jointly in many important applications, such as object recognition, based on Image rendering, and image (or video) editing. However, these two types of problems have often been studied independently, and until recently no one has begun to study them together. Such as L. Quan, J. Wang, P. Tan, and L. Yuan. Image-based modeling by joint segmentation. International Journal of Computer Vision (IJCV'07). [0003] Multi-view stereo restoration (depth restoration) technology (Multi-View Stereo, or MVS) can be used to calculate depth and 3D geometric informatio...

AI Technical Summary

Problems solved by technology

This method relies heavily on the accuracy of motion estimation, and it is difficult to obtain high-quality segmentation results, especially when severe occlusions occur.

[0009] In addition, two-dimensional motion segmentation also needs to obtain the calculation of motion and segmentation, both of which are "chicken and egg" problems, that is to say, inaccurate motion estimation will lead to inaccurate segmentation, which in turn will affect the inaccurate segmentation. Accurate will cause inaccurate motion calculation

Images