Convolutional Neural Network Video Frame Interpolation

The patent replaces traditional mechanical correspondence estimation methods with a deep learning-based CNN system. The neural network automatically learns feature correspondences and motion patterns from data, substituting the manual feature matching and warping process with an intelligent system that handles color changes, lighting variations, and motion blur more effectively.

The patent transforms the interpolation problem from direct pixel warping to a multi-stage process involving phase-based decomposition, frequency domain transformation, and hierarchical CNN processing. By changing the parameter space from spatial domain to frequency domain and applying multi-scale analysis, the system achieves higher accuracy in challenging scenes.

The patent segments the image into different frequency components through phase-based decomposition, processing each frequency band separately through the CNN. This segmentation allows the system to handle various motion ranges effectively while maintaining high interpolation quality, as different frequency components capture different aspects of the motion and texture information.

The patent moves the processing from the spatial domain to the frequency domain through phase-based decomposition and Fourier transformation. This dimensional change enables the system to handle a wider range of motion by analyzing and synthesizing image content in the frequency domain, where motion patterns can be more effectively separated and processed.

The patent applies preliminary action by first training the CNN on lower resolution versions of the images before progressing to full-resolution training. This gradual approach allows the network to learn basic patterns and structures first, establishing a stable foundation that accelerates subsequent training at higher resolutions and improves overall training stability.

The patent implements a nested training structure where lower resolution training is embedded within the overall training process, which then feeds into higher resolution training stages. Each resolution level is nested within the previous level, creating a hierarchical training pipeline that builds upon previously learned features at coarser scales.