Virtual viewpoint generation method and device

Abstract

The invention provides a virtual viewpoint generation method and device. The method comprises the following steps: according to a homography matrix determined by each checkerboard image shot by each binocular camera, carrying out homography transformation on a multi-view target image collected by each binocular camera to obtain a horizontal multi-view image; inputting the horizontal multi-view image into the trained parallax estimation model to obtain a parallax image; weighting the parallax image according to the control factors corresponding to different positions, and performing forward mapping according to the weighted parallax image and the horizontal multi-viewpoint image to generate a virtual viewpoint image; wherein the parallax estimation model is determined after each virtual image and dense real parallax are input into a full convolutional network for training. According to the virtual viewpoint generation method provided by the embodiment of the invention, the generation efficiency of the virtual viewpoint can be improved.

Description

technical field [0001] The present application relates to the technical field of image processing, and in particular to a method and device for generating a virtual viewpoint. Background technique [0002] The real world is three-dimensional, but the current mainstream display devices are still two-dimensional. 3D display, especially naked-eye 3D display, has received more and more attention. Naked-eye 3D display requires dense viewpoint images, and there are many difficulties in dense viewpoint acquisition with binocular camera arrays, such as synchronous adjustment between binocular camera arrays, binocular camera calibration and attitude calculation, and data storage and transmission. Therefore, in practical applications, a small number of real stereo cameras are usually used to collect sparse viewpoints, and dense viewpoints are generated by virtual viewpoint generation methods. [0003] Virtual viewpoint generation is the key to promoting the development of 3D display...

AI Technical Summary

Problems solved by technology

[0004]However, when generating virtual viewpoints of real scenes, it is necessary to collect multi-viewpoint images through binocular camera arrays and obtain parallax images, while DIBR is collecting multi-viewpoints When processing images, it is necessary to perform time-consuming calibration and attitude calculation on the binocular camera to correct the epipolar line of the multi-viewpoint image, and the speed of obtaining the parallax image by DIBR is slow, and the real-time performance cannot be guaranteed, resulting in low generation efficiency of virtual viewpoints.

Images