Motion Vector Refinement for TAA Ghosting in Temporal Supersampling

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Temporal Anti-aliasing (TAA) techniques suffer from ghosting artifacts around moving object boundaries due to mismatched motion vectors, leading to lower quality anti-aliasing when frames are rendered at native resolution.

Innovation Solution

A mixed low precision convolutional neural network is used for temporally amortized supersampling, generating residual motion vectors for pixels without initial motion information, and a residual motion refinement network is trained to enhance motion data accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If TAA is used to generate temporally stable anti-aliased images, then anti-aliasing quality is improved, but ghosting artifacts appear around moving object boundaries due to motion vector errors

Engineering Contradiction:

Improveanti-aliasing qualityVSAvoidghosting artifacts

Core Design Contradiction:

Manufacturing precisionVSObject-affected harmful factors

Solution Approach 1:

The system uses feedback by comparing the warped previous frame with the current frame at pixel level, calculating difference values that indicate motion vector errors. This feedback loop allows the system to identify and correct ghosting artifacts by adjusting motion vectors based on the detected discrepancies between frames.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent replaces traditional mechanical motion vector generation methods with a neural network-based system. The neural network learns to predict and refine motion vectors by analyzing frame differences, substituting the conventional renderer-generated velocity vectors with AI-enhanced vectors that reduce ghosting artifacts.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Manufacturing precision

If frames are rendered at native resolution to improve anti-aliasing quality, then image sharpness is improved, but render time increases

Engineering Contradiction:

Improveimage sharpnessVSAvoidrender time

Core Design Contradiction:

Manufacturing precisionVSProductivity

Solution Approach 1:

The system transitions from spatial supersampling (rendering at higher resolution) to temporal supersampling by accumulating and warping samples across multiple frames. This dimensional shift allows achieving high-quality anti-aliasing through temporal accumulation rather than spatial resolution, significantly reducing render time while maintaining image sharpness.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

3Manufacturing precision

If motion vectors are refined to eliminate ghosting artifacts, then anti-aliasing quality is improved, but computational complexity increases

Engineering Contradiction:

Improvemotion vector accuracyVSAvoidcomputational complexity

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The system performs preliminary action by pre-calculating and storing difference values between frames before motion vector refinement. This preprocessing step allows the neural network to efficiently refine motion vectors using pre-computed data, reducing the computational complexity during the actual rendering and motion vector refinement process.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS12367549B2Motion vector refinement for temporally amortized supersampling

Publication Date: 2025.07.22 INTEL CORP

AI summary

A residual network is used to predict a set of residual motion vectors that provide additional motion data for portions of the frame for which motion vectors are not provided, such as animated textures, mirrored/reflected objects, and/or moving objects without motion information.