Kernel Splatting Network for Temporally Amortized Supersampling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Temporal Anti-aliasing (TAA) techniques suffer from ghosting artifacts due to frame-to-frame changes in visibility and errors in motion vectors, leading to lower quality supersampled images when rendering at lower spatial resolutions.
Innovation Solution
A kernel splatting network combined with a low precision convolutional neural network is used to perform temporally amortized supersampling, enhancing rendering performance and image quality by aligning samples more accurately across frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If Temporal Anti-aliasing (TAA) accumulates temporally supersampled images to improve rendering quality, then image quality is improved, but ghosting artifacts occur due to frame-to-frame changes in visibility and motion vector errors
Solution Approach 1:
The patent introduces an intermediary mechanism (temporal supersampling buffer and alignment operations) between frame accumulation and final image generation. This intermediary layer aligns samples from different frames using motion vectors and visibility masks before accumulation, preventing ghosting artifacts while maintaining image quality improvement.
Solution Approach 2:
The patent performs preliminary alignment and filtering of temporal samples before accumulation. By pre-aligning samples using motion vectors and applying visibility masks to correct for frame-to-frame changes, the system prevents ghosting artifacts from forming during the accumulation process.
2Stability of the object's composition
If TAA warps sample history using motion vectors to align frames, then temporal alignment is improved, but errors in motion vectors cause mismatched samples and ghosting
Solution Approach 1:
The patent applies visibility masks as a protective measure before accumulation. These masks pre-identify and correct for frame-to-frame visibility changes, cushioning against the potential harm of motion vector errors by preventing mismatched samples from being accumulated in the first place.
Solution Approach 2:
The patent uses visibility masks that provide feedback information about frame-to-frame visibility changes. This feedback allows the system to adjust and correct sample alignment, compensating for motion vector errors and improving sample matching accuracy.
3Productivity
If rendering at lower spatial resolutions is used to improve rendering efficiency, then productivity is improved, but image quality deteriorates without proper supersampling
Solution Approach 1:
The patent transitions from spatial-only supersampling to temporal-spatial supersampling by accumulating samples across multiple frames in the temporal dimension. This allows the system to render at lower spatial resolutions while maintaining or improving image quality through temporal accumulation of multiple lower-resolution samples.
Solution Approach 2:
The patent maintains continuous accumulation of supersampled images across multiple frames. By continuously accumulating temporal samples and performing alignment operations, the system sustains high image quality throughout the rendering process even when operating at lower spatial resolutions.
Data Source
AI summary
One embodiment provides a graphics processor comprising a set of processing resources configured to perform a supersampling anti-aliasing operation via a mixed precision convolutional neural network. The set of processing resources include circuitry configured to receive, at an input block of a neural network model, a set of data including previous frame data, current frame data, jitter offset data, and velocity data, pre-process the set of data to generate pre-processed data, provide pre-processed data to a feature extraction network of the neural network model and an output block of the neural network model, process the first pre-processed data at the feature extraction network via one or more encoder stages and one or more decoder stages, output tensor data from the feature extraction network to the output block, and generate an anti-aliased output frame via the output block based on the current frame data and the tensor data output from the feature extraction network.