TIP Frame Prediction With Motion Vector Limits and Hole Filling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hole filling approaches for temporally interpolated picture (TIP) frame prediction in video coding require significant computing resources and have unbounded execution times, making them incompatible with hardware coders due to latency and computational strain, and motion vectors with unrestricted magnitudes pose additional challenges.
Innovation Solution
Implement motion vector magnitude restriction and hole filling techniques that limit motion vectors to the same superblock or a set of superblocks containing the hole, using scan order processing without epochs, to facilitate efficient TIP frame prediction in hardware coders.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing hole filling approaches for TIP frame prediction are used, then prediction accuracy is improved, but computing resources required increase significantly and execution time becomes unbounded
Solution Approach 1:
The patent divides the image into blocks and processes holes within specific block boundaries. By segmenting the hole filling process to only consider motion vectors within the same block or adjacent blocks rather than the entire image, the computational complexity is significantly reduced while maintaining prediction accuracy for the current block.
Solution Approach 2:
The patent applies local quality by restricting hole filling to use only motion vectors from the same block or adjacent blocks rather than searching the entire image. This localized approach maintains prediction accuracy for each block while reducing the overall computational burden and bounding the execution time.
2Measurement precision
If existing hole filling approaches are used, then prediction accuracy is improved, but execution time becomes unbounded making them incompatible with hardware coders
Solution Approach 1:
By segmenting the hole filling process to operate within block boundaries and adjacent blocks only, the patent creates a bounded execution time that is compatible with hardware coder requirements. The segmented approach ensures that hole filling completes within a finite number of operations rather than running indefinitely.
Solution Approach 2:
The patent applies partial action by performing hole filling only for blocks that actually contain holes and only searching within a limited scope (same block and adjacent blocks). This partial approach is sufficient for maintaining prediction accuracy while ensuring finite execution time, avoiding the excessive action of searching the entire image.
3Adaptability or versatility
If motion vectors with unrestricted magnitudes are used, then prediction flexibility is improved, but additional challenges are posed for hardware implementation
Solution Approach 1:
The patent applies local quality by restricting motion vector search to the same block or adjacent blocks only. This localized restriction maintains sufficient prediction flexibility for most cases while making hardware implementation feasible through bounded search ranges. The local quality approach trades some extreme prediction flexibility for practical hardware implementability.
Data Source
AI summary
Hardware-friendly approaches to generated reference frame prediction include motion vector magnitude restriction and hole filling. Motion vector magnitude restriction generally refers to the restriction of motion vector for prediction with a generated reference frame to a same block or a set of blocks including the same block within a portion of the generated reference frame. Hole filling generally refers to the identification of available motion vectors for a portion of a generated reference frame and their use for other cells of that portion for which motion vectors are unavailable. The disclosed motion vector magnitude restriction and hole filling approaches enable effective generated reference frame prediction in a hardware coder use case by, in relevant part, preventing the use of a motion vector that points outside of a generated reference frame portion stored within a working buffer or cache and avoiding unbounded execution times otherwise arising from epoch-based hole filling approaches.