Importance-Sampled Denoising and Upscaling Network for TAA Ghosting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Temporal Anti-aliasing (TAA) techniques in graphics rendering suffer from ghosting artifacts due to frame-to-frame visibility changes and motion vector errors, leading to lower quality anti-aliased images.
Innovation Solution
A combined denoising and upscaling network with importance sampling is employed, utilizing a mixed low precision convolutional neural network for temporally amortized supersampling to enhance image quality while reducing render time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If TAA accumulates temporally supersampled images with warped sample history, then anti-aliasing quality is improved, but ghosting artifacts occur due to visibility changes and motion vector errors
Solution Approach 1:
The patent extracts and removes mismatched historical samples from the accumulation buffer when visibility changes are detected. By identifying and eliminating these problematic samples before they contribute to the final image, the system prevents ghosting artifacts while preserving the benefits of temporal accumulation for stable anti-aliasing.
Solution Approach 2:
The patent implements a feedback mechanism that uses visibility information and motion vectors to monitor and evaluate the quality of accumulated samples. When mismatches are detected through this feedback loop, the system adjusts the accumulation process by removing problematic samples, thereby maintaining anti-aliasing quality without ghosting artifacts.
2Manufacturing precision
If high resolution rendering is performed to achieve image quality, then image quality is improved, but render time increases
Solution Approach 1:
The patent merges temporal anti-aliasing with supersampling techniques by accumulating multiple samples both spatially and temporally. This combination allows the system to achieve high-quality anti-aliased images at reduced resolution, effectively merging the quality benefits of high-resolution rendering with the efficiency of lower-resolution temporal accumulation.
Solution Approach 2:
The patent transitions from purely spatial sampling to temporal-spatial sampling by accumulating samples across time dimensions. This dimensional change allows the system to achieve high-quality anti-aliasing effects without requiring high spatial resolution at each individual frame, thereby reducing overall render time while maintaining image quality.
Data Source
AI summary
An apparatus to facilitate combined denoising and upscaling network with importance sampling in a graphics environment is disclosed. The apparatus includes set of processing resources including circuitry configured to: receive, at an input of a density map neural network, a sampled signal of a current frame and a reconstructed sample of the current frame; output, from the density map neural network, a prediction of a density map of samples based on the input of the current frame; provide the density map of samples to a sampler; reproject the density map of samples to a next frame; and apply the reprojected density map of samples to the next frame to generate a next sampled signal.