Unlock instant, AI-driven research and patent intelligence for your innovation.

Wide Baseline Multi-View Video Synthesis Method Based on Convolutional Neural Networks

A technology of convolutional neural network and multi-viewpoint video, applied in neural learning methods, biological neural network models, neural architectures, etc., can solve problems such as the inability to synthesize high-quality views, achieve reduced parameter volume, strong continuity, and good results Effect

Active Publication Date: 2020-11-06
NANKAI UNIV
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to solve the problem that existing viewpoint synthesis methods cannot synthesize high-quality views under wide baseline conditions, and propose a method that can simultaneously utilize high-dimensional depth and texture information for viewpoint synthesis

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Wide Baseline Multi-View Video Synthesis Method Based on Convolutional Neural Networks
  • Wide Baseline Multi-View Video Synthesis Method Based on Convolutional Neural Networks
  • Wide Baseline Multi-View Video Synthesis Method Based on Convolutional Neural Networks

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0020] The specific implementation manners of the present invention will be further described in detail below in conjunction with the accompanying drawings. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0021] refer to figure 1 , which represents the model structure based on convolutional neural network, the wide-baseline multi-view video synthesis method is as follows:

[0022] a. Input the depth map and texture map of the reference viewpoint: 4 frames of images corresponding to two reference viewpoints at the same time are obtained from the video, and the texture maps of the reference viewpoint view1 and view2 are I 1 and I 2 , the depth map is D 1 and D 2 . like figure 1 As shown, the two texture maps I 1 and I 2 The first branch that forms a 6×H×W tensor input network is the texture branch; the two depth maps D 1 and D 2 The second branch that forms a 2×H×W tensor input net...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

A Wide Baseline Multi-View Video Synthesis Method Based on Convolutional Neural Networks. The purpose of this method is to extract the features of surrounding viewpoint texture maps and depth maps and perform multi-view video content synthesis by predictively separable spatially adaptive convolution kernels. This method designs a new convolutional neural network model. The model uses two branches to complete the extraction and fusion of depth map and texture map features, and uses a set of independent sub-networks for separable spatial adaptive convolution kernel prediction, and finally combines the obtained convolution kernel with the input texture map Do local convolution to complete view synthesis. This method makes reasonable use of depth information to guide the prediction of adaptive convolution kernels, and can maintain good results in wide baselines and complex scenes.

Description

technical field [0001] The invention belongs to the technical field of image and video processing, in particular to a method for synthesizing multi-viewpoint video based on a convolutional neural network. Background technique [0002] Multi-view synthesis technology is dedicated to using the content of surrounding camera viewpoints to render and synthesize target viewpoint content. 3D stereoscopic display has attracted much attention from industry and academia because it can bring users an immersive visual experience. The real visual experience is at the cost of a large amount of data content, which brings great challenges to the acquisition and transmission of video in reality. Multi-viewpoint video is the mainstream representation method of 3D video. Multi-viewpoint synthesis technology can reduce the amount of 3D video data while ensuring that users can watch high-quality images at any viewpoint. Although many multi-view synthesis techniques have been proposed, the qual...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): H04N13/161G06N3/04G06N3/08
CPCH04N13/161G06N3/084G06N3/045
Inventor 卢少平王榕
Owner NANKAI UNIV