3D Scene Rendering with Learned Multi-Resolution Denoising
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Path-tracing for rendering 3-D scenes is computationally expensive due to the need for a large number of samples to reduce noise, making it impractical for devices with limited resources, and existing denoising algorithms like guided filters require structural information that may not be readily available.
Innovation Solution
A method using machine learning models to derive enhanced guide channels from initial guide channels, allowing for efficient denoising by approximating noisy images as a function of these enhanced channels, which can be applied at different resolutions to produce denoised images, reducing computational load and memory requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the number of samples is increased to reduce noise in path-traced images, then the quality of the rendered image is improved, but the computational cost increases significantly
Solution Approach 1:
The patent segments the rendering process into two distinct stages: a coarse path-tracing pass that generates noisy images at lower computational cost, and a separate denoising pass that processes the noisy images using learned denoising models. This segmentation allows the system to achieve high-quality results without requiring the prohibitively high sample counts that would be needed if path-tracing alone were used.
Solution Approach 2:
The patent introduces noisy rendered images as an intermediary representation between the 3D scene and the final high-quality image. Instead of directly generating high-quality images through expensive path-tracing, the system first creates noisy images that capture essential scene information, then uses denoising models to transform these intermediaries into high-quality final images.
2Use of energy by moving object
If denoising algorithms are applied to reduce noise without increasing samples, then computational cost is reduced, but image quality may deteriorate due to blurring or loss of detail
Solution Approach 1:
The patent transforms the denoising problem from operating directly on pixel values to operating in a feature space defined by guide channels. By changing the parameter space from raw pixel intensities to structured feature representations (depth, surface normals, ambient occlusion), the denoising process can preserve important structural information while removing noise, avoiding the blurring and detail loss that plagues traditional denoising methods.
Solution Approach 2:
The patent combines multiple types of guide channel information (depth maps, surface normals, ambient occlusion) into a composite feature representation that enriches the noisy images with structural context. This composite information acts as a guide for the denoising process, enabling it to distinguish between noise and genuine image features, thereby preserving quality while reducing computational requirements.
3Measurement precision
If guide channels are used to guide the denoising process, then structural information is preserved, but the complexity of the rendering pipeline increases
Solution Approach 1:
The patent implements guide channel generation using standard rendering techniques that can serve multiple purposes: they guide the denoising process, provide structural information for quality preservation, and can be reused across different rendering scenarios. The same guide channels that assist denoising also provide valuable information for other rendering operations, reducing overall system complexity despite the added denoising step.
4Measurement precision
If high-resolution path-tracing is performed to achieve detailed images, then image quality is improved, but memory requirements and computational resources increase
Solution Approach 1:
The patent segments the high-resolution rendering task into a low-resolution noisy image generation stage followed by a denoising stage. The coarse path-tracing pass operates at lower resolution with reduced memory requirements, generating noisy images that retain essential scene information. The subsequent denoising process enhances detail without requiring the prohibitive memory resources that would be needed for direct high-resolution path-tracing.
Solution Approach 2:
The patent creates a lower-resolution copy of the scene through noisy path-tracing, which serves as a surrogate for the full high-resolution rendering. This copied representation captures the essential structure and content of the scene at reduced computational and memory cost, which is then enhanced through denoising to recover fine details without requiring the original high-resolution data.
Data Source
Figure 1
Figure 2
Figure 3~5
AI summary
A method of rendering an image of a 3-D scene is provided. The method comprises: rendering (110) a noisy image at a first resolution; obtaining (112) one or more initial guide channels at the first resolution, and obtaining (140) one or more corresponding initial guide channels at a second resolution. The second resolution may be the same resolution as, or a higher resolution than, the first resolution. When the two resolutions are the same, the one or more initial guide channels at the first resolution and the one or more corresponding initial guide channels at the second resolution may be provided by a single set of initial guide channels. Enhanced guide channels are derived from the initial guide channels using one or more machine learning models. For each of a plurality of local neighbourhoods, the method comprises: calculating (120) the parameters of a denoising model that approximates the noisy image (in the local neighbourhood) as a function of the one or more enhanced guide channels (at the first resolution), and applying (130, 150) the calculated parameters to the one or more enhanced guide channels (at the second resolution), to produce a denoised image at the second resolution.