Joint CNN Denoising and Supersampling for Ray-Traced Graphics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Temporal Anti-aliasing (TAA) techniques in graphics processing suffer from ghosting artifacts due to mismatched motion vectors, and existing methods for denoising and supersampling are computationally expensive and often separate, leading to inefficiencies in rendering quality and time.
Innovation Solution
A single mixed-precision convolutional neural network (CNN) is used to jointly perform denoising and supersampling, combining temporal upsampling with TAA to enhance image quality and reduce rendering time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If separate analytical denoising and supersampling methods are used, then processing can be performed with simpler individual operations, but the overall rendering time increases and computational efficiency decreases
Solution Approach 1:
The patent combines separate denoising and supersampling operations into a single unified neural network model. This integrated approach processes both tasks simultaneously in one computational pass, eliminating the need for sequential processing of multiple separate analytical methods, thereby reducing overall rendering time while maintaining processing quality.
Solution Approach 2:
The unified neural network model performs multiple functions (denoising and supersampling) within a single processing architecture. This multi-functional approach allows the system to handle both operations with one optimized computational pipeline rather than requiring separate specialized processors for each task.
2Stability of the object's composition
If Temporal Anti-aliasing with motion vector warping is used, then temporal stability is improved, but ghosting artifacts occur due to mismatched samples
Solution Approach 1:
The patent replaces the traditional mechanical motion vector warping approach with a data-driven neural network-based system. Instead of relying on potentially inaccurate motion vectors to warp and align historical frames, the neural network learns optimal alignment and denoising strategies from data, automatically handling temporal stabilization without introducing ghosting artifacts.
Solution Approach 2:
The neural network incorporates feedback mechanisms to continuously optimize the alignment and denoising process. By learning from accumulated processing results and adjusting its operations accordingly, the system maintains temporal stability while avoiding the ghosting artifacts that plague fixed motion vector approaches.
3Manufacturing precision
If traditional denoising and supersampling pipelines are used, then processing steps remain simple and separate, but image quality suffers and rendering time increases
Solution Approach 1:
The patent merges denoising and supersampling into a single integrated neural network processing step. This unified approach achieves superior image quality by coordinating both operations simultaneously rather than sequentially, while reducing rendering time by eliminating redundant processing passes and intermediate storage operations.
Data Source
AI summary
Joint denoising and supersampling of graphics data is described. An example of a graphics processor includes multiple processing resources, including a least a first processing resource including a pipeline to perform a supersampling operation; and the pipeline including circuitry to jointly perform denoising and supersampling of received ray tracing input data, the circuitry including first circuitry to receive input data associated with an input block for a neural network, second circuitry to perform operations associated with a feature extraction and kernel prediction network of the neural network, and third circuitry to perform operations associated with a filtering block of the neural network.