Method and apparatus of motion vector derivations in immersive video coding

Abstract

Methods and apparatus of processing 360-degree virtual reality images are disclosed. According to one method, a system that maps a motion vector in a 2D frame into a 3D space and then applies rotationof sphere to adjust motion vector in the 3D space. The rotated motion vector is then mapped back to the 2D frame for processing 360-degree virtual reality images. According to another method, a system that derives motion vector from displacement of viewpoint is disclosed. According to yet another method, a system that applies scaling to adjust motion vector is disclosed. The motion vector scalingcan be performed in the 3D space by projecting a motion vector from a 2D frame to a 3D sphere. After motion vector scaling, the scaled motion vector is projected back to the 2D frame. Alternatively,the forward projection, motion vector scaling and inverse projection can be combined into a single function.

Description

[0001] priority statement [0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 523,883 filed on June 23, 2017 and U.S. Provisional Patent Application No. 62 / 523,885 filed on June 23, 2017. The aforementioned US Provisional Patent Application is hereby incorporated by reference in its entirety. technical field [0003] The present invention relates to image / video processing or encoding and decoding of 360-degree virtual reality (virtual reality, VR) images / sequences. In particular, the present invention relates to deriving motion vectors for three-dimensional (3D) content in different projection formats. Background technique [0004] 360-degree video, also known as immersive video, is an emerging technology that can provide "the feeling of being there". A feeling of immersion is achieved by surrounding the user with a wrap-around scene covering the panoramic view, especially a 360-degree panorama. The "sense of being there" can be furthe...

AI Technical Summary

Problems solved by technology

If conventional inter prediction is applied to 2D frames transformed from 3D space, using motion estimation/motion compensation techniques will not work properly because...

Images