Multi-camera dynamic scene 3D (three-dimensional) rebuilding method based on joint optimization

Abstract

The invention discloses a multi-camera dynamic scene 3D (three-dimensional) rebuilding method based on joint optimization, which can be used for partitioning a space-time consistent state, is robust and does not need prior knowledge of a background. According to the method, multi-view shooting depth recovery and static / dynamic dual-layer partitioning which participate in shooting can be solved simultaneously, free and independent movement is realized without any accurate static / dynamic camera, and dynamic scenes shot by using certain cameras can be processed. The method comprises the following steps of: I, initializing a depth map at a moment T by using a synchronous video frame which skips M cameras at each moment T; and II, performing space-time consistence depth optimization and static / dynamic dual-layer partition, and implementing space-time consistence optimization iteratively to realize high-quality dynamic 3D rebuilding. According to the method, accurate static / dynamic partition is not required; and the method has robustness specific to error partition information generated in complex scenes, and has very high application values in the fields of 3D images, 3D animations, augmented reality, motion capture and the like.

Description

technical field [0001] The invention relates to stereo matching and depth restoration methods, in particular to a multi-camera dynamic scene 3D reconstruction method based on joint optimization. Background technique [0002] The dense depth restoration technology of video is one of the basic technologies in the field of computer mid-level vision, and it has important applications in many fields such as 3D modeling, 3D imaging, augmented reality and motion capture. These applications usually require high accuracy and spatiotemporal consistency of depth restoration results. [0003] The difficulty of dense depth recovery technology for video lies in the fact that for static and dynamic objects in the scene, the recovered depth values ​​have high precision and spatiotemporal consistency. Although the current depth restoration technology for static scenes has been able to restore high-precision depth information, the natural world is full of moving objects. For dynamic objects ...

AI Technical Summary

Problems solved by technology

The methods of Larsen and Lei respectively use the method of belief propagation in the space-time domain and the method of 3D smoothing in the time domain to optimize the depth value, making these methods not robust enough to deal with serious errors in optical flow estimation.

Images