Continuous Block Tracking for Video Motion Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video compression algorithms face limitations in inter-prediction, particularly due to exhaustive search methods that are computationally prohibitive and may not always find the best motion vectors, especially in rapidly changing data, and model-based compression schemes struggle with object segmentation and tracking.
Innovation Solution
The implementation of continuous block tracking (CBT) and trajectory-based continuous block tracking (TB-CBT) methods that allow for efficient navigation of the prediction search space by tracking data blocks of varying sizes and using frame-to-frame tracking results to generate motion vectors for input blocks, even across non-adjacent reference frames, without requiring feature or object detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If exhaustive search methods are used for motion estimation, then prediction accuracy may be improved, but computational complexity becomes prohibitive
Solution Approach 1:
The patent applies preliminary action by performing frame-to-frame tracking to generate initial motion vector estimates before the actual motion estimation process. These preliminary motion vectors serve as starting points that guide the subsequent refinement process, avoiding the need for exhaustive search while maintaining accuracy. The tracking step prepares the system in advance by establishing plausible motion hypotheses that constrain the search space.
Solution Approach 2:
The patent segments the motion estimation process into distinct stages: frame-to-frame tracking to generate initial estimates, followed by refinement steps. This segmentation allows the system to handle the complex task of motion estimation in manageable steps, where each stage contributes to the final accuracy without requiring the entire process to be computationally exhaustive.
2Adaptability or versatility
If conventional block-based motion estimation is used, then compatibility with standard codecs is maintained, but efficiency in rapidly changing data is reduced
Solution Approach 1:
The patent introduces dynamics by adapting the motion estimation approach based on the characteristics of the video data. For rapidly changing data, the system uses frame-to-frame tracking with refined search strategies that are more responsive to motion than conventional block-based methods. This dynamic adaptation allows the system to maintain codec compatibility while improving efficiency for specific content types.
Solution Approach 2:
The patent changes key parameters of the motion estimation process, including the search space definition and motion vector refinement strategies. By modifying these parameters based on data characteristics, the system achieves better encoding efficiency for rapidly changing content while maintaining compatibility with standard codec frameworks through the preservation of fundamental block-based operations.
3Productivity
If model-based compression schemes are used, then compression efficiency may be improved, but object segmentation and tracking challenges increase
Solution Approach 1:
The patent extracts and utilizes tracking information from adjacent frames without requiring full object segmentation. By taking out only the necessary motion information from frame-to-frame analysis and using it to guide block-based prediction, the system achieves compression efficiency benefits while avoiding the complexity of complete object segmentation and tracking.
Solution Approach 2:
The patent introduces an intermediary approach where frame-to-frame tracking results serve as a mediator between the video data and the block-based motion estimation process. This intermediary tracking information guides the motion estimation without requiring the system to perform complex object segmentation, thus achieving compression efficiency while avoiding segmentation complexity.
Data Source
AI summary
Continuous block tracking (CBT) tracks macroblock locations over reference frames to produce better inter-predictions than conventional block-based motion estimation/compression. CBT includes frame-to-frame tracking, estimating motion from a frame to a previous frame, and continuous tracking, related frame-to-frame motion vectors to block tracks. Frame-to-frame tracking may include block based or hierarchical motion estimations. CBT combined with enhanced predictive zonal search may create unified motion estimation. Accumulated CBT results may form trajectories for trajectory-based CBT predictions. Metrics measuring continuous track and motion vector quality can assess relative priority of CBT prediction against non-tracker-based predictions and to modify encoding choices. Continuous tracks can be analyzed for goodness-of-fit to translational motion models, with outliers removed from encoding. Translational motion models can be extended to entire frames in adaptive picture type selection. Outputs from CBT used in look-ahead processing, via look-ahead tracking, may provide rate control and scene change detection for the current frame being encoded.