Parallel Low Resolution Pipeline for Video Motion Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The computational complexity of determining motion vectors for inter-frame prediction techniques in video encoding pipelines is high, making real-time or near real-time encoding and transmission challenging due to the extensive search required for candidate motion vectors.
Innovation Solution
A low resolution pipeline is introduced in parallel with the main pipeline to perform motion estimation searches ahead of time, providing initial candidate inter-frame prediction modes and statistics, such as luma histograms and zero vector sums of absolute differences, to reduce the search space and computational load for the main pipeline.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If inter-frame prediction techniques are used to encode image data, then transmission bandwidth and memory usage are reduced, but computational complexity increases due to extensive motion vector search
Solution Approach 1:
The patent segments the motion estimation process into two distinct pipelines: a low-resolution pipeline that performs initial motion estimation on downsampled image data, and a main pipeline that refines motion vectors using the results from the low-resolution pipeline. This segmentation allows the computationally intensive full-resolution search to be divided into a coarse initial search and a fine refinement stage, reducing overall computational complexity while maintaining encoding efficiency.
Solution Approach 2:
The low-resolution pipeline performs preliminary motion estimation before the main pipeline processes full-resolution data. By conducting an initial search at reduced resolution and generating candidate motion vectors in advance, the system prepares a narrowed search space for the main pipeline, significantly reducing the computational burden of the subsequent full-resolution motion estimation while still achieving accurate motion compensation.
2Measurement precision
If full resolution motion estimation is performed, then motion vector accuracy is improved, but encoding time increases making real-time processing difficult
Solution Approach 1:
The motion estimation process is segmented into a low-resolution stage that quickly identifies candidate motion vectors and a main pipeline stage that refines these vectors at full resolution. This segmentation enables the system to achieve accurate motion vectors without performing exhaustive full-resolution searches on all possible candidate blocks, thereby reducing encoding time while maintaining motion vector accuracy.
Solution Approach 2:
The low-resolution pipeline performs preliminary motion estimation to generate candidate motion vectors before the main pipeline executes full-resolution refinement. This preliminary action narrows down the search space significantly, allowing the main pipeline to focus computational resources only on promising candidates, thus achieving accurate motion vectors within real-time encoding constraints.
3Productivity
If a parallel low resolution pipeline is introduced, then computational load of main pipeline is reduced, but device complexity increases
Solution Approach 1:
The patent merges the low-resolution pipeline and main pipeline into a unified encoding architecture where the low-resolution pipeline feeds candidate motion vectors to the main pipeline. This merging allows the system to leverage the computational benefits of reduced-resolution processing while maintaining the quality of full-resolution encoding, achieving high encoding efficiency without requiring completely separate independent systems.
Solution Approach 2:
The low-resolution pipeline acts as an intermediary between the input image data and the main full-resolution pipeline. It processes data at reduced resolution and generates candidate motion vectors that serve as input to the main pipeline, mediating the computational load and enabling the main pipeline to focus on refinement rather than exhaustive search, thus improving overall encoding efficiency.
Data Source
AI summary
System and method for improving operational efficiency of a video encoding pipeline used to encode image data. In embodiments, the video encoding pipeline includes a low resolution pipeline that includes a low resolution motion estimation block, which generates downscaled image data by reducing resolution of the image data and determines a low resolution inter-frame prediction mode by performing a motion estimation search using the downscaled image data and previously downscaled image data. The video encoding pipeline also includes a main pipeline in parallel with the low resolution pipeline that includes a motion estimation block, which determines a candidate inter-frame prediction mode based at least in part on the low resolution inter-frame prediction mode, and a mode decision block, which determines a first rate-distortion cost associated with the candidate inter-frame prediction mode and determines prediction mode used to prediction encode the image data based at least in part on the first rate-distortion cost.