Virtual viewpoint image generation method based on generation type confrontation network

Abstract

The invention relates to a virtual viewpoint image generation method based on a generation type confrontation network. The method comprises the following steps: first step, making a data set: obtaining an image pair necessary for training the generation type confrontation network; constructing a model: the structures used by a generator and a discriminator are that batch normalization layers BatchNorm and nonlinear operation unit ReLU activation functions are connected behind convolutional layers, all convolutional layers use a 4*4 convolution kernel size, the step size is set as 2, the lengthand width are both reduced to a half of the original length and width when downsampling is performed on a feature image, the length and width are both amplified to twice of the original length and width during upsampling, and a Dropout layer sets a Dropout rate as 50%; the RelU activation function is LeakyReLu; defining the loss; and training and testing the model.

Description

technical field [0001] The invention belongs to the field of stereo vision and deep learning, and relates to a method for generating virtual viewpoint images using a generative confrontation network model. Background technique [0002] In daily life, when human beings look at objects with both eyes, the horizontally arranged left and right eyes respectively observe objects from slightly different angles, so the images captured by the left and right eyes are slightly different. Due to the perception of the human visual system and the brain, the left and right eye views are fused in the brain, so that humans can feel the obvious depth from their subtle differences, and establish the corresponding relationship between features, and combine the physical points of the same space in different images. The image points correspond to produce a three-dimensional concept of the observed things. [0003] Binocular stereo vision is an important form of machine vision, and its most basic...

AI Technical Summary

Problems solved by technology

However, past research, such as the generation method of virtual viewpoint images based on depth maps, often relies on prior information such as depth maps and disparity maps of two viewpoint images.

But this is not applicable to more general natural images when depth information or disparity information is unknown

Images