Joint Neural Denoising of Surfaces and Volumes for Low-Sample Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional denoisers struggle to effectively denoise images containing both surface geometry and volumes, especially when low sample counts are used, as feature guides for volumes are harder to define and noise-free guides are not readily available, leading to inadequate results and increased image noise.
Innovation Solution
A joint neural denoising approach that separates surface and volume components, using spatio-temporal neural denoisers to process each component individually and combine them with learned weights and denoised transmittance, enabling real-time denoising from low sample count renderings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional denoisers are applied to images containing both surfaces and volumes, then processing speed is maintained, but image quality deteriorates due to increased noise and artifacts
Solution Approach 1:
The patent segments the denoising process into two distinct components: surface denoising and volume denoising. Surface data and volume data are processed separately using specialized denoisers optimized for each type, rather than applying a single conventional denoiser to the entire image. This segmentation allows each denoiser to focus on the specific characteristics of its data type, reducing artifacts and improving overall image quality.
Solution Approach 2:
The patent introduces transmittance data as an intermediary element that guides the combination of denoised surface and volume components. The transmittance map indicates the contribution of volume versus surface to each pixel, enabling intelligent blending of the two denoised components. This intermediary facilitates the integration of separately processed surface and volume data while maintaining physical accuracy and reducing artifacts.
2Productivity
If low sample counts are used for real-time rendering, then rendering speed is improved, but image quality deteriorates due to increased noise
Solution Approach 1:
The patent segments the rendering output into separate surface and volume components that are denoised independently. This allows specialized denoisers to efficiently process each component with low sample counts, maintaining rendering speed while improving the quality of each component through targeted denoising strategies.
Solution Approach 2:
The patent employs temporal denoising that uses feedback from previous frames to improve current frame quality. By accumulating and utilizing temporal information across multiple frames, the system can achieve higher quality results from low sample counts while maintaining real-time rendering speeds through temporal coherence.
3Shape
If feature-based surface denoisers are applied to volumes, then surface edge preservation is improved, but volume denoising quality deteriorates due to lack of appropriate feature guides
Solution Approach 1:
The patent segments the denoising process into surface-specific and volume-specific operations. Surface denoisers with edge-preserving capabilities are applied only to surface data where they are effective, while volume denoisers optimized for volumetric data are applied to volume data. This prevents the misapplication of surface-based feature guides to volume data where they would be ineffective.
Solution Approach 2:
The patent applies different denoising qualities and methods to different parts of the image based on their nature. Surface regions receive denoising optimized for surface geometry and edges, while volume regions receive denoising optimized for volumetric properties and transmittance. This local customization of denoising strategy ensures optimal quality for each region type.
Data Source
AI summary
Denoising images rendered using Monte Carlo sampled ray tracing is an important technique for improving the image quality when low sample counts are used. Ray traced scenes that include volumes in addition to surface geometry are more complex, and noisy when low sample counts are used to render in real-time. Joint neural denoising of surfaces and volumes enables combined volume and surface denoising in real time from low sample count renderings. At least one rendered image is decomposed into volume and surface layers, leveraging spatio-temporal neural denoisers for both the surface and volume components. The individual denoised surface and volume components are composited using learned weights and denoised transmittance. A surface and volume denoiser architecture outperforms current denoisers in scenes containing both surfaces and volumes, and produces temporally stable results at interactive rates.