Motion Compensated Temporal Processing Complexity Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Motion-compensated temporal analysis in video processing is computationally and memory-intensive, making it challenging for applications with power and memory constraints, such as real-time video encoding, due to the need for extensive motion estimation and buffering of multiple reference pictures.
Innovation Solution
The implementation of spatiotemporal sub-sampling techniques, including temporal down-sampling of input pictures and reduction of reference pictures, to reduce computational and memory complexity while maintaining high performance in motion parameter generation and filtering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If motion-compensated temporal analysis is performed using multiple reference pictures, then filtering performance and compression efficiency are improved, but computational complexity and memory requirements increase significantly
Solution Approach 1:
The patent extracts and processes only the most essential information from reference pictures through spatiotemporal sub-sampling. Instead of using full-resolution reference pictures, the system extracts motion parameters and essential visual information at reduced spatial and temporal resolutions, thereby reducing computational complexity while maintaining filtering performance.
Solution Approach 2:
The patent segments the temporal processing by dividing the video sequence into groups of pictures (GPBs) and further into sub-GPBs. This segmentation allows motion-compensated temporal filtering to be applied selectively at different temporal resolutions, reducing overall computational complexity while maintaining reliability in critical regions.
2Measurement precision
If motion estimation is performed on all reference pictures, then motion parameter accuracy is improved, but processing time and computational load increase
Solution Approach 1:
The patent implements periodic action by performing motion estimation at reduced temporal frequency. Instead of estimating motion parameters for every picture, the system performs motion estimation periodically on selected reference pictures and uses these estimates for multiple subsequent predictions, thereby reducing processing time while maintaining acceptable accuracy.
Solution Approach 2:
The patent applies partial action by performing motion estimation only on a subset of reference pictures rather than all available references. The system selectively applies motion estimation to pictures that provide the most valuable information, accepting partial coverage in exchange for reduced processing time.
3Measurement precision
If multiple reference pictures are buffered for motion compensation, then prediction accuracy is improved, but memory requirements increase
Solution Approach 1:
The patent applies local quality by maintaining different qualities of reference pictures in memory based on their utility. Instead of uniformly storing all reference pictures at full resolution, the system maintains high-quality references for critical regions and lower-quality or sub-sampled references for less critical areas, optimizing memory usage while preserving prediction accuracy where needed.
4Device complexity
If temporal down-sampling is applied to reduce complexity, then computational load is reduced, but motion parameter accuracy may deteriorate
Solution Approach 1:
The patent implements dynamics by adaptively adjusting the temporal down-sampling factor based on scene complexity and motion characteristics. In low-motion scenes, higher down-sampling is applied to reduce computational load, while in high-motion or complex scenes, the system reduces down-sampling to preserve motion parameter accuracy, creating a dynamic balance between complexity and precision.
Data Source
AI summary
A method and system for reduced complexity motion compensated temporal processing for pre-analysis purposes. The processing complexity is reduced by reducing the number of pictures processed for analysis, reducing the number of references used for analysis, and/or spatially subsampling input pictures.