Bidirectional Optical Flow Estimation With Image Pyramids
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing optical flow estimation methods suffer from large operational overheads, low efficiency, and limited robustness and generalization due to high parameter scales and the need for unidirectional estimation, which hinders real-time performance and effective bidirectional flow estimation.
Innovation Solution
A bidirectional optical flow estimation method utilizing image pyramids and recursive calls of a pre-trained optical flow estimation model, combined with forward warping and channel stacking, to reduce model parameters and improve estimation accuracy and robustness across different scales.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If an optical flow estimation model with large parameter scale is used, then estimation accuracy can be improved, but training overheads increase and operational efficiency decreases
Solution Approach 1:
The patent segments the optical flow estimation process into multiple passes operating on image pyramids. Instead of using a single large model, it divides the estimation into coarse-to-fine stages (different pyramid levels), where each stage uses a smaller model on downsampled images, progressively refining the flow field. This segmentation reduces the computational burden of each individual model invocation while maintaining overall accuracy through multi-scale analysis.
Solution Approach 2:
The patent introduces the dimension of image scale resolution by constructing image pyramids with multiple resolution levels. By transforming the problem from a single-resolution estimation to a multi-resolution hierarchy, it enables smaller models to capture global motion patterns at coarse levels while finer levels handle detailed local motions, thereby improving efficiency without sacrificing accuracy.
2Reliability
If the model is trained on a specific picture scale, then training convergence is improved, but robustness to different scales is reduced
Solution Approach 1:
The patent makes the optical flow estimation system universal across different image scales by implementing a pyramid-based architecture. The same model can be applied at multiple resolution levels, and the system adapts to different input scales automatically through the hierarchical structure. This multi-functionality allows the model trained at one scale to generalize to other scales by processing them through the same pyramid pipeline.
Solution Approach 2:
By adding the resolution dimension through image pyramids, the system transforms scale-specific training into scale-invariant operation. The model processes images at multiple predefined resolutions, enabling it to handle various input scales robustly without requiring separate training for each scale, thus achieving both convergence reliability and scale adaptability.
3Device complexity
If unidirectional optical flow estimation is performed, then model complexity is reduced, but bidirectional flow estimation requires multiple runs decreasing efficiency
Solution Approach 1:
The patent merges forward and backward optical flow estimation into a single unified processing pipeline. By implementing bidirectional estimation within the same multi-pass pyramid framework, it estimates both forward (I0→I1) and backward (I1→I0) flows simultaneously during the same model invocations, eliminating the need for separate unidirectional runs and thereby improving efficiency while maintaining manageable model complexity.
Solution Approach 2:
The patent ensures continuous useful action by maintaining the same optical flow estimation model and processing pipeline for both forward and backward directions. The model performs both directional estimations continuously within each pyramid level without interruption or re-initialization, maximizing operational efficiency while keeping the model architecture relatively simple through reuse.
Data Source
AI summary
A bidirectional optical flow estimation method and apparatus are provided. The method includes acquiring a target image pair of which optical flow is to be estimated, and constructing an image pyramid for each target image in the target image pair respectively, and performing bidirectional optical flow estimation using a pre-trained optical flow estimation model based on the image pyramid, to obtain bidirectional optical flow between the target images. An optical flow estimation module in the optical flow estimation model is recursively called to perform the bidirectional optical flow estimation sequentially based on images of respective layers in the image pyramid according to a preset order, forward warping towards middle processing is performed on an image of a corresponding layer of the image pyramid before each call of the optical flow estimation module, and an image of an intermediate frame obtained by the forward warping towards middle processing is inputted into the optical flow estimation module. With the disclosure, the efficiency and generalization of bidirectional optical flow estimation can be improved, and model training and optical flow estimation overheads can be reduced.