Probabilistic Mesh Motion Estimation for Smooth Video Motion Fields
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing motion estimation techniques in video coding systems face challenges in achieving efficient compression and motion continuity, particularly with complex models like affine motion models, leading to issues such as the aperture problem and discontinuities in motion fields.
Innovation Solution
Implementing probabilistic mesh motion estimation (MME) that introduces inter-probability in the motion model, uses master and slave nodes, and employs non-fully connected meshes for mesh continuity preservation, addressing the aperture problem and improving compression efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If complex motion models (e.g., affine motion model) are used for motion estimation, then motion accuracy is improved, but computational complexity and device complexity increase
Solution Approach 1:
The video frame is divided into multiple triangular mesh regions instead of using a single complex model for the entire frame. Each triangle is defined by three control points with motion vectors, allowing local motion estimation to be simpler while collectively capturing complex motion patterns across the frame. This segmentation reduces the computational burden on any single processing unit while maintaining overall motion accuracy.
Solution Approach 2:
The patent implements a hybrid approach where the motion model dynamically adapts between block-based and mesh-based methods. The encoder can switch between using traditional block matching for simple regions and probabilistic mesh motion estimation for regions with complex motion, optimizing the balance between accuracy and computational complexity based on actual content requirements.
2Device complexity
If traditional block matching is used for motion estimation, then device complexity is reduced, but motion continuity and compression efficiency deteriorate
Solution Approach 1:
The patent replaces the rigid rectangular block structures with triangular mesh elements that can better conform to curved motion boundaries and object contours. The triangular geometry allows for more flexible representation of motion fields, particularly for rotating or non-translational motion, thereby improving compression efficiency without requiring overly complex processing.
Solution Approach 2:
The patent combines elements of both block-based and mesh-based motion estimation approaches. It uses control points at triangle vertices that can be processed similarly to block motion vectors, while the triangular connectivity provides mesh-based continuity. This merging allows implementation with moderate device complexity while achieving improved compression efficiency through better motion representation.
3Measurement precision
If more motion vectors are used to improve motion field accuracy, then motion precision is improved, but computational load and processing time increase
Solution Approach 1:
Instead of uniformly distributing motion vectors across the entire frame, the patent places control points at triangle vertices with higher density in regions exhibiting complex motion patterns. The probabilistic mesh generation algorithm adaptively refines the mesh in areas with high motion variability while using coarser meshes in stable regions, optimizing the distribution of computational resources to match actual motion complexity.
Solution Approach 2:
The patent uses a probabilistic approach where not all triangular elements require full motion estimation. The system can selectively apply detailed mesh motion estimation to only those regions where it is most beneficial, while using simpler models or fewer control points in other areas. This partial application of the complex model maintains motion field accuracy where needed while reducing overall computational load.
Data Source
AI summary
Disclosed herein are systems, devices and methods for probabilistic mesh motion estimation. The systems, devices and methods can involve an encoder, appliance, or other device for processing the video data to estimate motion in one or more locations of the video data using probabilistic mesh motion estimation based on a motion model. The encoder can introduce inter-probability in the motion model for appearing areas in the video data in conjunction with using the probabilistic mesh motion estimation. The encoder can introduce a probability that regions in the video data are intra-regions or inter-regions in the motion model as weights to indicate a confidence of finite elements being inter-associated. The encoder can use master and slave nodes and over-splitting for mesh continuity preservation. The encoder can use a non fully connected mesh applicable to video coding standards.