Neural Supersampling with Sample Reliability for TAA Ghosting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Temporal Anti-aliasing (TAA) techniques in graphics rendering suffer from ghosting artifacts due to frame-to-frame visibility changes and motion vector errors, leading to lower quality upsampling when rendering at native resolution.
Innovation Solution
A mixed low precision convolutional neural network is employed for temporally amortized supersampling, incorporating sample reliability information to improve image quality and reduce blurring, while rendering at lower resolution for performance gains.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If Temporal Anti-aliasing (TAA) is used to generate temporally stable anti-aliased images, then temporal stability is improved, but ghosting artifacts appear around moving object boundaries due to frame-to-frame visibility changes and motion vector errors
Solution Approach 1:
The system performs preliminary actions by accumulating supersampled data over multiple frames before final image generation. The neural network is trained in advance to recognize and correct ghosting artifacts, allowing it to preliminarily prepare the correct mapping between accumulated samples and current frame pixels, thereby preventing ghosting rather than merely reacting to it.
Solution Approach 2:
The neural network acts as an intermediary between the accumulated supersampled data and the final anti-aliased image. It processes the mapped sample history and current frame information, using sample reliability weights as intermediate calculations to determine which samples to trust, thereby mediating the conflict between temporal accumulation and artifact prevention.
2Manufacturing precision
If rendering is performed at native resolution with TAA, then image quality is improved, but render time increases
Solution Approach 1:
The system performs partial supersampling by accumulating a limited number of supersamples over multiple frames rather than performing full supersampling at native resolution in a single frame. This partial action approach achieves acceptable image quality while significantly reducing the computational burden and render time compared to traditional supersampling methods.
Solution Approach 2:
The system dynamically adjusts the number of samples accumulated based on scene complexity, motion magnitude, and reliability metrics. The neural network dynamically weights sample contribution based on motion vector accuracy and visibility changes, allowing the system to adaptively balance image quality and render time in real-time rather than using a fixed supersampling rate.
3Manufacturing precision
If the renderer accumulates supersampled data temporally, then anti-aliasing quality is improved, but mismatched sample history occurs due to visibility and shading changes
Solution Approach 1:
The system implements feedback through the neural network that continuously evaluates sample reliability based on motion vector accuracy, visibility changes, and shading consistency. This feedback mechanism identifies mismatched samples and adjusts their weights accordingly, creating a closed-loop system that self-corrects for visibility and shading changes between frames.
Solution Approach 2:
The system changes parameters by introducing sample reliability weights that modulate the contribution of accumulated samples based on their validity. When visibility or shading parameters change between frames, the neural network adjusts the weight parameters of corresponding samples, effectively filtering out mismatched data while preserving valid historical samples for anti-aliasing.
Data Source
AI summary
A graphics processor is provided that includes circuitry configured to receive, at an input block of a neural network model, a set of data including previous frame data, current frame data, velocity data, and jitter offset data. The neural network model is configured to generate a denoised, supersampled, and anti-aliased output image based on reliability metrics computed based on sample distribution data for samples within the current frame data.