Neural Temporal Supersampling for TAA Ghosting Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Temporal Anti-aliasing (TAA) techniques suffer from ghosting artifacts around moving object boundaries due to frame-to-frame changes in visibility and errors in motion vectors, leading to lower quality upsampling when rendering at native resolution.
Innovation Solution
A mixed low precision convolutional neural network is used for temporally amortized supersampling, incorporating sample reliability information to improve image quality and reduce blurring, while rendering at lower spatial resolution for a performance boost.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If Temporal Anti-aliasing (TAA) accumulates temporally supersampled images, then anti-aliasing quality is improved, but ghosting artifacts appear around moving object boundaries
Solution Approach 1:
The system performs preliminary actions by accumulating temporally supersampled images before final rendering, and also discards outdated frame data in advance to prevent ghosting artifacts from forming in the first place
Solution Approach 2:
The system changes parameters by adjusting the accumulation window size and temporal weighting factors dynamically, allowing it to optimize between anti-aliasing quality and ghosting reduction based on scene motion characteristics
2Stability of the object's composition
If TAA warps previous frame using motion vectors, then temporal alignment is improved, but mismatches occur due to motion vector errors
Solution Approach 1:
The system uses feedback by comparing accumulated samples with current frame data and adjusting temporal weighting accordingly, allowing it to detect and correct mismatches caused by motion vector errors
Solution Approach 2:
The system applies partial temporal accumulation rather than full frame warping, using only the necessary portion of historical data to maintain alignment while avoiding the propagation of motion vector errors
3Manufacturing precision
If rendering is performed at native resolution, then image quality is improved, but render time increases
Solution Approach 1:
The system transitions from purely spatial rendering to temporal-spatial supersampling, accumulating samples across multiple frames to achieve native resolution quality at lower effective render rates
Solution Approach 2:
The system performs preliminary supersampling accumulation over multiple frames before final output, distributing the computational workload over time to reduce peak render time while maintaining quality
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A graphics processor is provided that includes circuitry configured to receive, at an input block of a neural network model, a set of data including previous frame data, current frame data, velocity data, and jitter offset data. The neural network model is configured to generate a denoised, supersampled, and anti-aliased output image based on reliability metrics computed based on sample distribution data for samples within the current frame data.