An Intelligent Virtual Reference Frame Generation Method

Abstract

The invention discloses a method for generating an intelligent virtual reference frame. The method comprises the following steps: according to the characteristics of the virtual reference frame in the time domain and the reference frame between viewpoints, using an alignment step based on viewpoint synthesis to learn the parallax relationship, and according to the parallax relationship, the viewpoint The inter-reference frame is aligned with the time-domain virtual reference frame to generate an offset inter-view reference frame; for the time-domain virtual reference frame and the offset inter-view reference frame, the complementary information between the two is fused to generate a high-quality Virtual reference frame; build a virtual reference frame generation network based on viewpoint synthesis, and use two MSE loss functions to constrain the training of the generation network. The present invention provides a high-quality reference for the current coding frame, thereby improving the prediction accuracy and further improving the coding efficiency.

Description

technical field [0001] The invention relates to the fields of 3D video coding and deep learning, in particular to an intelligent virtual reference frame generation method. Background technique [0002] Multi-viewpoint video is a typical representation of 3D video, which can provide viewers with an immersive stereoscopic feeling. In order to improve the compression efficiency of multi-viewpoint video, the international standard organization JCT-VC has launched the multi-viewpoint plus depth video coding standard 3D-HEVC. For multi-view video coding, 3D-HEVC not only uses HEVC (High Efficiency Video Coding) coding technology to eliminate temporal redundancy and spatial redundancy within the same view, but also introduces a variety of inter-view coding technologies to eliminate inter-view redundancy. In addition, the overall coding efficiency is effectively improved. [0003] The data volume of multi-view video is much larger than that of 2D video, which brings great challeng...

AI Technical Summary

Problems solved by technology

[0006] Compared with the successful application of deep learning in the field of 2D video coding, there is still a lack of multi-view video coding methods based on deep learning technology; the existing inter-view predictive coding tools for 3D-HEVC do not make full use of the disparity relationship between adjacent viewpoints. Coding efficiency still needs to be further improved

Images