Adaptive Sub-sampling Motion Estimation for Video Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing motion estimation techniques require a large number of pixels to determine motion vectors, especially for high-speed motion, leading to increased computational complexity and bandwidth requirements, while previous methods that reduce pixel numbers often compromise image quality by discarding useful information.
Innovation Solution
The method involves subsampling pixels in a predefined pattern within the search area, with higher resolution maintained near the center and increased sub-sampling towards the edges, allowing for reduced pixel storage and processing requirements while minimizing loss of image detail by adapting to the direction and speed of motion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a large search area is used to support high-speed motion estimation, then the maximum motion vector size is increased, but the number of pixels required increases and bandwidth requirements increase
Solution Approach 1:
The search area is divided into multiple zones with different sub-sampling rates. The central region uses full or high resolution while peripheral regions use coarser sub-sampling. This segmentation allows the system to support large motion vectors (high speed capability) while reducing the total number of pixels processed, thereby resolving the contradiction between speed support and pixel quantity.
Solution Approach 2:
Different regions of the search area are assigned different qualities (resolution levels). The central region maintains higher quality for accurate motion estimation, while peripheral regions use lower quality sub-sampled data. This local quality differentiation enables the system to handle high-speed motion with large search areas without requiring uniformly high resolution across the entire area, thus reducing overall pixel requirements while maintaining speed capability.
2Device complexity
If pixels are subsampled to reduce the number of pixels, then computational complexity and bandwidth requirements are reduced, but image quality deteriorates due to loss of useful information
Solution Approach 1:
The sub-sampling is applied non-uniformly across the search area, with finer sampling in the central region and coarser sampling at the periphery. This ensures that the most important region (center) retains sufficient detail for accurate motion estimation, while the less critical peripheral regions use coarser sampling to reduce computational complexity and bandwidth. This local quality differentiation resolves the contradiction by maintaining image quality where needed while reducing complexity where possible.
Solution Approach 2:
The sub-sampling pattern is adapted dynamically based on the motion characteristics and search area requirements. The system can adjust the sub-sampling rate in different regions based on the actual motion magnitude and direction, ensuring that pixels are not oversampled in regions with little motion information while maintaining sufficient detail in regions with significant motion. This dynamic adaptation reduces overall computational complexity while preserving necessary image quality.
3Measurement precision
If full search is performed across the entire previous image, then the most accurate motion vector is found, but the computational complexity increases significantly
Solution Approach 1:
The full search area is segmented into zones with different processing intensities. The central region undergoes more thorough evaluation with higher effective resolution, while peripheral regions use coarser evaluation. This segmentation allows the system to maintain high measurement precision for the most likely motion vectors (which tend to be in the central region) while reducing computational complexity by applying less intensive processing to peripheral regions where exact precision is less critical.
Solution Approach 2:
Instead of applying full computational effort uniformly across the entire search area, the system applies partial action with varying intensity. The central region receives more computational attention (excessive action relative to need), while peripheral regions receive less computational action. This differential approach maintains motion vector accuracy where it matters most while significantly reducing overall computational complexity compared to uniform full-search processing.
Data Source
AI summary
Systems and methods of determining motion vectors, such as for video encoding, are disclosed. In one example, motion vectors are determined for a current frame, using sampled pixel information from a reference frame. Sampled pixel information is obtained using a sampling pattern. The sampling pattern, in one example, includes subsampling pixels at different rates for horizontal and vertical directions. The subsampling rate can differ, based on an amount of motion represented by a matching block (e.g., the farther a match is found away from an origin of the block, the more subsampling can be done). In another example, a full pixel resolution is maintained proximal an original location of the block; as distance increases in one or more directions, subsampling can begin and/or increase. Sampled pixels can be stored. Interpolation of the sampled pixels can be performed and the sampled and resulting interpolated pixels can be used for comparison.