Panoramic Video Decoding with 3D Projection Center Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies struggle to efficiently handle the complex motion models in panoramic videos, which are often captured with moving cameras, leading to inefficiencies in decoding and image quality.
Innovation Solution
A video decoding apparatus and method that utilize inter frame prediction by computing differences between projection centers of reference and current frames in a 3D coordinate system, transforming reference frames to align with current frames, and using global motion compensation to improve decoding efficiency and quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If conventional video decoding technologies are used for panoramic videos, then the decoding process can be simplified, but the inter frame prediction efficiency deteriorates due to the complex three-dimensional nature and camera movements
Solution Approach 1:
The patent transforms the reference frame from its original two-dimensional representation to a three-dimensional spherical coordinate system that matches the panoramic video's projection geometry. This dimensional transformation enables accurate motion compensation by aligning the reference frame's projection center with the current frame's projection center, thereby improving inter frame prediction efficiency without excessive complexity increase
Solution Approach 2:
The patent modifies the projection parameters of the reference frame by calculating and applying a projection center transformation. This involves changing the angular and radial parameters of the reference frame's spherical coordinates to match the current frame's projection center, enabling effective motion compensation for panoramic videos captured with moving cameras
2Manufacturing precision
If motion compensation is applied to compensate for camera movements in panoramic videos, then the image quality improves, but the computational load increases
Solution Approach 1:
The patent divides the panoramic video frame into multiple blocks or regions, and applies motion compensation selectively to each block based on its specific motion characteristics. This segmentation approach improves image quality by accurately compensating for local camera movements while reducing the overall computational load by processing smaller independent regions rather than the entire frame at once
Solution Approach 2:
The patent applies motion compensation only to the necessary portions of the reference frame that correspond to actual motion regions in the current frame. By identifying and processing only the relevant blocks that require compensation rather than applying the transformation to the entire frame, the method reduces computational load while maintaining image quality in the motion-affected areas
3Measurement precision
If the projection center transformation is applied to align reference frames, then the inter frame prediction accuracy improves, but the processing time increases
Solution Approach 1:
The patent performs projection center transformation and reference frame alignment in advance during the encoding phase, storing the transformed reference frames in a buffer. This preliminary action ensures that when decoding occurs, the alignment is already completed, thereby improving inter frame prediction accuracy without significantly increasing real-time processing time during playback
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system and method for providing motion compensation in decoding of a panoramic video. The panoramic video comprises a sequence encoded of image frames. Using inter frame prediction an image frame of a video is expressed in terms of at least one of neighboring image frames. In panoramic video image frames are mapped on a spherical viewing area. In the method the motion caused by camera movements is compensated by computing a difference between a reference frame and a current frame. The computed difference is then used when re-constructing the current frame, which may be placed to an image buffer and used as a reference frame for other frames.