Motion Estimation Apparatus Using Segmented Coarse and Fine Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video processing methods for motion estimation are calculation-intensive and time-sensitive, especially at higher frame and resolution sizes, due to the need for precise motion vectors with integer-pixel precision, which increases complexity and bandwidth requirements.
Innovation Solution
The method involves determining correlation values at a predetermined granularity to produce a motion vector with initial precision, and then refining it to higher precision by considering blocks displaced by half or quarter pixels, using techniques like SAD or SSAD, and interpolating pixel values for finer granularity searches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If motion estimation is performed with integer-pixel precision using whole pel granularity searching, then motion vector precision is sufficient for basic video processing, but calculation complexity and processing time increase significantly at higher resolutions and frame rates
Solution Approach 1:
The patent segments the motion estimation process into two distinct stages: a coarse search stage that operates at whole-pixel granularity to identify a preliminary motion vector, and a fine search stage that operates at sub-pixel granularity to refine the motion vector. This segmentation allows the system to avoid performing computationally intensive sub-pixel searches across the entire search area, thereby reducing overall processing complexity while maintaining high precision in the final motion vector.
Solution Approach 2:
The patent performs a preliminary coarse search at whole-pixel granularity before conducting the fine sub-pixel search. This preliminary action identifies a candidate motion vector that is sufficiently accurate for most purposes, and then uses this result to limit the scope of the subsequent fine search. By performing this preliminary action, the system avoids the computational burden of performing exhaustive sub-pixel searches across the entire search area, thus reducing processing complexity while maintaining precision.
2Measurement precision
If sub-pixel precision motion estimation is achieved by up-sampling frames to higher granulation, then motion vector precision increases, but bandwidth requirements and processing capacity requirements increase greatly
Solution Approach 1:
The patent segments the search process into coarse and fine stages, where the coarse search operates on the original resolution frames without up-sampling. This segmentation allows the system to obtain a preliminary motion vector from the coarse search, and then use this vector to guide a focused fine search that operates on up-sampled data only in the relevant region. This approach achieves sub-pixel precision while minimizing bandwidth requirements by avoiding full-frame up-sampling.
Solution Approach 2:
The patent applies partial up-sampling only to the necessary regions and stages of motion estimation rather than up-sampling entire frames. Specifically, the fine search stage performs up-sampling only for the candidate blocks identified in the coarse search stage, rather than up-sampling the entire reference frame. This partial action achieves the needed sub-pixel precision while significantly reducing the bandwidth and memory requirements compared to full-frame up-sampling approaches.
3Measurement precision
If exhaustive block searching is performed across the entire search area to ensure accurate motion vector determination, then measurement accuracy is maintained, but processing time becomes excessive for real-time video applications
Solution Approach 1:
The patent segments the search process into two phases: a rapid coarse search phase that covers the entire search area at whole-pixel granularity to identify candidate motion vectors, and a focused fine search phase that refines these candidates at sub-pixel granularity. This segmentation maintains measurement accuracy by thoroughly searching the entire area in the coarse phase, while reducing processing time by limiting the computationally intensive fine search to only the candidate blocks identified in the coarse phase, rather than performing fine searches across the entire search area.
Solution Approach 2:
The patent performs a preliminary coarse search across the entire search area before conducting the fine search. This preliminary action identifies the most likely candidate blocks that contain the true motion vector, allowing the subsequent fine search to be focused on a small subset of candidates rather than the entire search area. This preliminary action maintains measurement accuracy by ensuring the true motion vector is identified in the coarse phase, while dramatically reducing processing time by limiting the fine search scope.
Data Source
AI summary
Methods and apparatus for facilitating motion estimation in video processing are provided. In one embodiment, search block is defined within one frame. A relative location of a corresponding block in another frame with respect to the search block is determined based on comparative searching at a predetermined granularity to produce a motion vector for the search block with a first precision. Correlation values are determined with respect to the search block for the corresponding block and for one block or more blocks defined at relative locations of less than the predetermined granularity with respect to the corresponding block in different directions. A refined motion vector for the search block with a second higher precision is determined based on the relative location of the block having a selected correlation value that is selected from among the determined correlation values.