Procedural Shader Motion Vectors for TAA Ghosting Reduction

A neural network is introduced as an intermediary component that processes motion vectors and generates corrected correspondence data. The neural network takes as input the current frame, previous frame, and renderer-generated motion vectors, then outputs refined motion correspondence information that eliminates ghosting artifacts while preserving temporal stability.

The traditional renderer-generated motion vector system is replaced with a neural network-based correspondence detection system. Instead of relying on analytical motion vectors from the renderer, the system uses a trained neural network to learn and predict accurate correspondences between frames, substituting a data-driven approach for the analytical approach.

A neural network is trained in advance on a dataset of high-resolution images and corresponding motion vectors. This preliminary training phase allows the network to learn complex correspondence patterns, so that during actual rendering, the pre-trained network can quickly process frames and generate accurate supersampled images without requiring extensive computation at render time.

The system transitions from spatial upsampling alone to temporal-spatial upsampling by incorporating multiple frames and their correspondences. The neural network processes information across the temporal dimension, using historical frame data to inform current frame reconstruction, thereby achieving higher quality supersampling that leverages both spatial and temporal dimensions.