Video Sequence Temporal Consistency via Recursive Frame Splitting

The patent divides the video sequence into multiple segments or blocks of frames. Instead of minimizing the energy functional over the entire video sequence at once, the optimization is performed independently on each segment. This segmentation approach maintains temporal consistency within each segment while significantly reducing the computational burden compared to global optimization, thus resolving the contradiction between reliability (temporal consistency) and loss of time (computational time).

The patent performs preliminary computations such as motion estimation and feature detection on individual frames or small groups of frames before the actual energy minimization. By pre-computing motion fields and other intermediate results, the subsequent energy minimization requires fewer iterations and less computational time, thereby reducing the overall computational time while maintaining the temporal consistency through the preliminary structured preparation.

The patent segments the video sequence into multiple independent blocks for parallel processing. Each block can be optimized separately with reduced computational complexity, avoiding the need to handle the entire video sequence as a single large optimization problem. This segmentation directly addresses the contradiction by maintaining temporal consistency through structured segmentation while reducing computational complexity through distributed optimization.

The patent transforms the temporal dimension by processing frames in spatially distributed blocks rather than sequentially through the entire time sequence. This dimensional transformation allows parallel computation across multiple blocks, reducing the overall computational complexity from O(T) sequential operations to O(T/n) parallel operations where n is the number of blocks, while still ensuring temporal consistency within each block.

The patent applies segmentation to the computation of non-parametric motion models by dividing the video sequence into smaller blocks. Optical flow or other non-parametric motion estimation is computed independently on each block, reducing the computational resources required for each individual motion estimation while maintaining high motion accuracy within each block. The segmented approach allows parallel processing that significantly reduces overall computational resource usage.

The patent computes motion fields and performs energy minimization only on relevant regions or blocks where motion is significant, rather than processing the entire video sequence with full non-parametric motion models. This partial action approach maintains high motion accuracy in critical regions while reducing computational resources by avoiding unnecessary computations in static or less important regions.