A Neural Network Light Field Method Based on Joint Sampling Structure

A neural network and network technology, applied in the field of image processing, can solve the problems of speeding up view synthesis, reducing the amount of calculation, and reducing the network training time, so as to reduce the amount of sampling calculation, reduce the calculation time, and save the training time.

Active Publication Date: 2021-12-14
BEIJING UNIV OF POSTS & TELECOMM
View PDF10 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Aiming at the problems of long training time and slow calculation speed caused by repeated uniform sampling when the original NeRF method fits the spatial light field, the present invention proposes a neural network light field method based on a joint sampling structure, using rough network and fine network Sharing uniform sampling results and synergistically synthesizing joint sampling of new views reduces the amount of computation in the entire process, thereby reducing network training time and speeding up view synthesis

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A Neural Network Light Field Method Based on Joint Sampling Structure
  • A Neural Network Light Field Method Based on Joint Sampling Structure
  • A Neural Network Light Field Method Based on Joint Sampling Structure

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0044] The application environment for realizing the neural network light field of the present invention is as follows:

[0045] The neural network light field rendering algorithm program is written in Python language, and the neural network part is implemented based on the PyTorch framework. The data processing and logic of the program run on a single thread of the CPU, and the main bottleneck of the algorithm efficiency lies in the calculation of the ray sampling results by the neural network, which is operated by the GPU. The algorithm runs on the server, and the server needs to install CUDA (Compute Unified Device Architecture, Unified Computing Device Architecture) for inference training of the neural network.

[0046] The parameters involved in the neural network light field algorithm include two categories:

[0047] The first category is light sampling related parameters. In this example, the number of locations where each ray is input to the rough network for uniform...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention is a neural network light field method based on joint sampling structure, which is used for reconstruction of three-dimensional scenes. The method of the invention establishes a neural network light field with a joint sampling structure, uses pictures with calibrated camera parameters for training, and then uses the trained neural network light field to calculate the color of each pixel in the new view to generate a new view. The present invention improves the existing technical scheme of using rough and fine double-network sampling to fit the spatial light field, establishes a neural network light field with a joint sampling structure, cascades the rough network and the fine network, and the final color output is the output of the two networks co-generated results. The present invention no longer performs uniform sampling on delicate networks, reduces sampling calculation amount and calculation time, ensures view synthesis quality while reducing calculation amount, improves the speed of generating three-dimensional views, and saves training time of network models.

Description

technical field [0001] The invention belongs to the technical field of image processing, and relates to image-based three-dimensional scene acquisition, reconstruction and display, in particular to a method for fitting and displaying three-dimensional scene light information based on a neural network model of a joint sampling structure. Background technique [0002] Light Field (Light Field) rendering is a new convenient and dynamic technology for presenting 3D scenes. Light field rendering technology does not require professionals to design and reconstruct complex geometric models, nor does it require fine texture maps and lighting simulations. As long as a certain number of photos are taken at multiple viewpoints, the original pictures can be directly used at new viewpoints that have not been taken. Synthesize realistic scene views. The principle of light field rendering is to collect light information in the scene space as much as possible, and then use the light informa...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06T7/80G06T7/90G06T17/00G06K9/62G06N3/04G06N3/08
CPCG06T17/00G06T7/80G06T7/90G06N3/04G06N3/08G06T2207/10052G06T2207/20081G06T2207/20076G06F18/2321
Inventor 刘绍华李明豪
Owner BEIJING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap