Light field multi-plane representation reconstruction method and device based on neural network

A neural network and multi-plane technology, applied in the field of multi-plane representation reconstruction of light field based on neural network, can solve problems such as huge computing pipelines, incompatible representations of scenes, etc., to reduce system complexity and achieve good light field interpolation effect. Effect

Inactive Publication Date: 2020-05-08
TSINGHUA UNIV
View PDF3 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] However, existing light field reconstruction methods always focus on one of the problems but fail to solve the other, and t

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Light field multi-plane representation reconstruction method and device based on neural network
  • Light field multi-plane representation reconstruction method and device based on neural network
  • Light field multi-plane representation reconstruction method and device based on neural network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0025] Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

[0026] The method and device for reconstructing multi-plane representations of light fields based on neural networks according to the embodiments of the present invention will be described below with reference to the accompanying drawings. First, the method for reconstructing multi-plane representations of light fields based on neural networks will be described with reference to the accompanying drawings. .

[0027] figure 1 It is a flowchart of a method for reconstructing multi-plane representations of light fields based on neu...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a light field multi-plane representation reconstruction method and device based on a neural network, and the method comprises the following steps: generating a light field dataset through employing a scene of an open source code SceneNet; training an MPI reconstruction network according to the light field data set, performing sparse light field acquisition on the dynamic scene by using the camera array, and reconstructing an MPI for each viewpoint of a camera of the camera array by using the MPI reconstruction network; and filtering the alpha channel and the color vector of the reconstructed MPI in a time domain to form a target viewpoint MPI, selecting a target viewpoint MPI of a neighbor viewpoint for MPI fusion of a new viewpoint to be reconstructed, and generating a reconstructed image under the new viewpoint according to a rendering method of mixing transparency from front to back. According to the method, the system complexity can be greatly reduced, anda better light field interpolation effect can be realized in a space domain and an angle domain at the same time.

Description

technical field [0001] The invention relates to the technical field of computer vision, in particular to a method and device for reconstructing multi-plane representations of light fields based on neural networks. Background technique [0002] For sparsely sampled light fields, generating viewpoint-free images is a key problem in the field of computer vision. High-quality light field rendering can generate realistic views in real-time, and the computational complexity is independent of the complexity of the scene, whether it has non-Lambertian surfaces (such as jewelry, fur, glass and faces, etc.). Generating free viewpoints based on sparse light fields faces two technical challenges: large parallax and non-Lambertianity. [0003] However, existing light field reconstruction methods always focus on one of the problems but fail to solve the other, and these algorithms either have incompatible representations for the scene, or lead to huge computational pipelines after direct...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06T7/50G06T7/90G06T11/00G06N3/08
CPCG06T11/005G06T7/90G06T7/50G06N3/08
Inventor 刘烨斌钟源戴琼海
Owner TSINGHUA UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap