Real time adaptive generation method from double-viewpoint video to multi-viewpoint video

Abstract

The invention relates to video image processing. The invention provides a real time obtaining method of realizing a multi-viewpoint image with few cameras. A technical scheme employed in the invention is a real time adaptive generation method from a double-viewpoint video to a multi-viewpoint video. The real time adaptive generation method comprises the following steps: 1. carrying out spatial domain depth matching: (a) carrying out cross searching with a pixel point on a left viewpoint image as a center and determining a matched window; (b) carrying out cross searching to obtain a plurality of windows to be matched; (c) designing an energy function, substituting the matched window, the windows to be matched, and extra compensation information into the energy function for calculation, and obtaining a corresponding depth map point of a left viewpoint endowed by initial parallax; (d) carrying out region parallax refinement, according to continuity and consistency criterions, designing a rectification function, and correcting parallax with a distribution error in an initial parallax graph; 2. carrying out time domain depth matching; 3. generating an intermediate virtual viewpoint. The method mainly is applied to the video image processing.

Description

technical field [0001] The invention relates to the technical field of video and generation of intermediate virtual viewpoints based on left and right viewpoints, in particular, to a real-time self-adaptive generation method from dual-viewpoint video to multi-viewpoint video. Background technique [0002] At present, the real-time acquisition method and technology of multi-viewpoint stereoscopic images are still the unsolved bottleneck problem of multi-viewpoint autostereoscopic imaging technology. To collect stereoscopic images of multiple viewpoints in real time, it is common practice to use N cameras arranged in parallel to shoot the same scene at the same time, so as to obtain N images of different positions (viewpoints). But this method itself has many problems: (1) The system is bulky and inconvenient for real-time acquisition, so it can only be used for multi-view stereo image acquisition of fixed scenes. (2) Due to the need for a comfortable image, very high require...

AI Technical Summary

Problems solved by technology

But this method itself has many problems: (1) The system is heavy, and real-time acquisition is inconvenient, so it can only be used for multi-view stereo image acquisition of fixed scenes

(2) Due to the need for a comfortable image, a very high requirement is placed on the consistency of the optical parameters and imaging parameters of multiple cameras, which brings great difficulty to the calibration and adjustment of the optical system

(3) The amount of multi-view image data collected is too large, which is not conducive to compression, transmission and storage

Seitz proposed a principle framework algorithm that can be used to generate new viewpoint images, but in practical applications, the quality of the obtained images is not good due to the influence of image pixel matching accuracy

Moreover, this method is only suitable for the source image collected by the camera moving around the scene, and it is powerless to the image collected by the camera moving forward and backward.

Hu Zhiping of Dalian University of Technology proposed a method of generating new viewpoint images based on images collected by horizontal and vertical camera movement in her doctoral thesis "Image Feature Extraction Matching and New Viewpoint Image Generation Technology". This method uses dynamic programming matching method, the complexity is high, and the image generated by interpolation has to undergo post-transformation to obtain the final virtual viewpoint image

Both pre-transformation and post-transformation will have deviations, resulting in an image that is not at the real virtual viewpoint

Images