Ray Tracing CNN Denoising and Supersampling for Ghosting Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Temporal Anti-aliasing (TAA) techniques in graphics processing suffer from ghosting artifacts due to frame-to-frame visibility changes and motion vector errors, while separate network operations for denoising and supersampling are computationally costly and inefficient.
Innovation Solution
A single mixed-precision convolutional neural network (CNN) is used to jointly perform denoising and supersampling, combining temporal upsampling with TAA to enhance image quality and reduce rendering time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If separate network operations are used for denoising and supersampling, then each operation can be optimized independently, but the computational cost and processing time increase significantly
Solution Approach 1:
The patent combines separate denoising and supersampling network operations into a single unified neural network that performs both functions simultaneously. This merging reduces computational overhead and processing time while maintaining the quality benefits of both operations, directly resolving the contradiction between image quality and processing efficiency
Solution Approach 2:
The unified neural network is designed to perform multiple functions (denoising and supersampling) within a single model architecture. This multi-functional approach allows the system to achieve the benefits of both specialized operations without the computational cost of executing them separately, thereby improving productivity while preserving measurement precision
2Measurement precision
If TAA accumulates temporal samples to produce supersampled image, then anti-aliasing quality improves, but ghosting artifacts occur due to visibility changes and motion vector errors
Solution Approach 1:
The patent extracts and removes the harmful ghosting artifacts from the temporal accumulation process by using the unified neural network to identify and eliminate mismatched warped samples. The network selectively processes temporal samples to remove artifacts while preserving the beneficial anti-aliasing effects, thus resolving the contradiction between quality improvement and artifact generation
3Measurement precision
If renderer jitters camera to sample different coordinates for TAA, then anti-aliasing effectiveness improves, but mismatch between warped sample history and current pixel increases
Solution Approach 1:
The unified neural network incorporates feedback mechanisms to continuously evaluate and correct the alignment between warped sample history and current pixel coordinates. By using the network's predictive capabilities and comparing against actual current frame data, the system adjusts temporal sample weighting to maintain reliable alignment despite camera jitter, resolving the contradiction between sampling accuracy and temporal alignment reliability
Data Source
AI summary
Joint denoising and supersampling of graphics data is described. An example of a graphics processor includes multiple processing resources, including a least a first processing resource including a pipeline to perform a supersampling operation; and the pipeline including circuitry to jointly perform denoising and supersampling of received ray tracing input data, the circuitry including first circuitry to receive input data associated with an input block for a neural network, second circuitry to perform operations associated with a feature extraction and kernel prediction network of the neural network, and third circuitry to perform operations associated with a filtering block of the neural network.