Omnidirectional Video Motion Compensation Using Block Warping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing motion compensation techniques for omnidirectional videos are inadequate for handling strong geometrical distortions and non-uniform pixel densities, leading to inefficient encoding and decoding processes, especially in immersive display devices like Head Mounted Displays.
Innovation Solution
The method involves computing block corners and three-dimensional offsets using a block warping function and transformation, followed by motion compensation on both parametric and three-dimensional surfaces, and applying inverse transformations for accurate motion prediction and compensation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing motion compensation techniques are used for omnidirectional videos, then the encoding and decoding processes are simpler, but the coding efficiency is poor due to strong geometrical distortions and non-uniform pixel densities
Solution Approach 1:
The omnidirectional video frame is divided into multiple blocks, and motion compensation is performed independently for each block. This segmentation allows the system to handle geometrical distortions and non-uniform pixel densities on a local basis, improving coding efficiency without requiring complex global transformations.
Solution Approach 2:
The patent applies different motion compensation strategies to different blocks based on their local characteristics. By computing block-specific corners and offsets, the system adapts to local geometrical distortions and pixel density variations, thereby improving overall coding efficiency while maintaining manageable complexity through localized processing.
2Productivity
If block warping function and three-dimensional transformation are applied for motion compensation, then coding efficiency is improved by adapting to omnidirectional video layout, but the computational complexity increases
Solution Approach 1:
The patent pre-computes block corners and three-dimensional offsets using warping functions and transformations. By performing these computationally intensive operations in advance during the encoding phase, the system achieves better coding efficiency while the decoded video can be played back without requiring the same level of computational power in real-time decoding.
Solution Approach 2:
The system creates transformed copies of video blocks with corrected geometrical distortions and uniform pixel densities. These copied and transformed blocks are then used for motion compensation, allowing the system to achieve high coding efficiency by working with pre-transformed data rather than repeatedly performing complex transformations during decoding.
Data Source
AI summary
An improvement in the coding efficiency resulting from improving the motion vector compensation process of omnidirectional videos is provided, which uses a mapping f to map the frame F to encode to the surface S which is used to render a frame. The corners of a block on a surface are rectified to map to a coded frame which can be used to render a new frame. Various embodiments include rectifying pixels and using a separate motion vector for each group of pixels. In another embodiment, motion vectors can be expressed in polar coordinates, with an affine model, using mapped projection or an overlapped block motion compensation model.