Large-scale panoramic viewpoint synthesis method based on single-viewpoint RGB-D image

A technology of RGB-D and viewpoint synthesis, which is applied in the field of image processing, can solve the problems of composite viewpoint structure overlap, large-area hole filling, deformation and distortion, etc., and achieve satisfactory visual perception effect, complete structure and good visual perception effect

Active Publication Date: 2020-06-23
XI AN JIAOTONG UNIV
View PDF6 Cites 13 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] Finally, for the problems of large-scale hole filling, composite viewpoint structure overlap, deformation and distortion in the process of large-scale viewpoint synthesis based on single-viewpoint RGB-D images, etc.
For the above-mentioned problems in the process of large-scale view synthesis of single-view RGB-D images, there are still many deficiencies in the existing methods

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Large-scale panoramic viewpoint synthesis method based on single-viewpoint RGB-D image
  • Large-scale panoramic viewpoint synthesis method based on single-viewpoint RGB-D image
  • Large-scale panoramic viewpoint synthesis method based on single-viewpoint RGB-D image

Examples

Experimental program
Comparison scheme
Effect test

preparation example Construction

[0051] see figure 1 , the present invention is based on a large-scale panoramic viewpoint synthesis method of a single-viewpoint RGB-D image, comprising the following steps:

[0052] S1. Perform 3D-Warping on the single-frame color image and the corresponding depth image respectively;

[0053] First of all, for a single frame RGB-D image, that is, P 0 (RGB 0 ,Depth 0 ) for 3D-Warping, the principle is to use depth information and camera parameters to map the image of the known reference viewpoint to the imaging plane of the virtual camera, and through 3D-Warping, the virtual viewpoint and mask image with holes will be obtained.

[0054] S2. Filling of virtual viewpoint holes based on CNN convolutional neural network;

[0055] When the viewpoint is transformed from a reference viewpoint to a virtual viewpoint, a part of the region that is visible in the virtual viewpoint but invisible in the original reference viewpoint is revealed, resulting in a large amount of missing in...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a large-scale panoramic viewpoint synthesis method based on a single-viewpoint RGB-D image, and the method comprises the steps: firstly carrying out the 3D-Warping of an inputted single-frame color image and a corresponding depth image, and obtaining a virtual viewpoint image with a cavity and a mask image; taking the virtual viewpoint image with the cavity and the mask image as the input of a CNN neural network; wherein an output result of the network model is an image after virtual viewpoint hole filling, and then taking a newly generated virtual viewpoint in the horizontal direction as a new input, and obtaining a virtual viewpoint image at any position in the space through 3D-Warping and CNN neural network hole filling. According to the method, the quality of the virtual viewpoint is improved on the whole, and the method has important guiding significance for improving the synthesis of the large-parallax virtual viewpoint.

Description

technical field [0001] The invention belongs to the technical field of image processing, and in particular relates to a method for synthesizing large-scale panoramic viewpoints based on single-viewpoint RGB-D images. Background technique [0002] With the development of science and technology and the improvement of people's living standards, the visual experience brought by traditional two-dimensional video screens has become increasingly unable to meet people's viewing needs. Therefore, many research institutes and industrial circles have paid more and more attention to the improvement of the vividness of video screens, and omnidirectional video screens, free-viewpoint video screens, and interactive stereoscopic video screens have emerged as the times require. Among them, the free view point video screen has been widely used because it can meet the needs of users to watch the video screen from any point of view. In recent years, the rapid development of image acquisition t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06T5/00G06T7/50G06N3/04G06N3/08
CPCG06T5/005G06T7/50G06N3/08G06T2207/10004G06T2207/10024G06T2207/10028G06T2207/20081G06T2207/20084G06N3/045
Inventor 杨勐马勇郑南宁
Owner XI AN JIAOTONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap