Using machine learning to filter monte carlo noise from images

Abstract

A method of producing noise-free images is disclosed. The method includes using machine learning incorporating a filter to output filter parameters using the training images. The machine learning may include training a neural network. The filter parameters are applied to Monte Carlo rendered training images that have noise to generate noise-free images. The training may include determining, computing and extracting features of the training images; computing filter parameters; applying an error metric; and applying backpropgation. The neural network may be a multilayer perceptron. The machine learning model is applied to new noisy Monte Carlo rendered images to create noise-free images. This may include applying the filter to the noisy Monte Carlo rendered images using the filter parameters to create the noise-free images.

Description

RELATED APPLICATION INFORMATION[0001]This patent claims priority from Provisional Patent Application No. 62 / 155,104, filed Apr. 30, 2015, titled A LEARNING-BASED APPROACH FOR FILTERING MONTE CARLO NOISE which is included by reference in its entirety.GOVERNMENT INTERESTS[0002]This invention was made with Government support under Grant (or Contract) Nos. IIS-1321168 and IIS-1342931 awarded by the National Science Foundation. The Government has certain rights in the invention.NOTICE OF COPYRIGHTS AND TRADE DRESS[0003]A portion of the disclosure of this patent document contains material which is subject to copyright protection. This patent document may show and / or describe matter which is or may become trade dress of the owner. The copyright and trade dress owner has no objection to the facsimile reproduction by anyone of the patent disclosure as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyright and trade dress rights whatsoever....

AI Technical Summary

Benefits of technology

This patent is about improving computer graphics rendering by removing noise from and improving Monte Carlo rendered images. The patent describes methods for quickly rendering a noisy image with a few samples and then filtering it as a post-process to generate an acceptable, noise-free result. The main technical effect of the patent is to provide a more efficient and effective way to produce high-quality images using distributed effects such as global illumination and depth of field. The patent also describes various techniques for adaptively sampling and using additional scene information to improve the filtering process.

Problems solved by technology

Although an approximation to this integral can be quickly evaluated with just a few samples, the inaccuracy of this estimate relative to the true value appears as unacceptable noise in the resulting image.

The high cost of computing additional rays results in lengthy render times that negatively affect the applicability of MC renderers in modern film production.

However, a major challenge is how to exploit this additional information to denoise distributed effects, which requires setting the filter weights for all features (called “filter parameters” hereafter) so that noise is removed while scene detail is preserved.

The main drawback of these methods is that their error metrics are usually noisy at low sampling rates, reducing the accuracy of filter selection.

As a result, these methods produce images with over / under blurred regions.

Although these color-based methods are general and work on a variety of distributed effects, they need many samples to produce reasonable results.

At low sampling rates, they generate unsatisfactory results on challenging scenes.

Although this method handles general distributed effects, it suffers from dimensionality.

The main problem the aforementioned approaches, which constitute the state of the art, is that they weight each filter term through either heuristic rules and / or an error metric which is quite noisy at low sampling rates.

Thus, they are not able to robustly estimate the appropriate filter weights in challenging cases.

In addition, Jakob et al. have a method that, while not utilizing neural networks, performs learning through expectation maximization to find the appropriate parameters of a Gaussian mixture model to denoise photon maps, a different but related problem.

Images