Virtual Reference Pictures for Residual Reduction in Inter-Picture Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies face challenges in reducing residual data after inter-prediction, leading to increased coded bit size despite improvements in coding efficiency through motion compensation.
Innovation Solution
Generating more accurate reference pictures using neural-network processing to improve video compression by creating virtual reference pictures from decoded reference pictures, which are then used for inter-picture prediction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If motion compensation is used for inter-picture prediction, then coding efficiency is improved, but residual data increases leading to larger coded bit size
Solution Approach 1:
The patent generates virtual reference pictures in advance from existing decoded reference pictures before they are needed for prediction. This preliminary generation of enhanced reference pictures allows the system to start with better prediction material, reducing the residual that needs to be encoded later in the process.
Solution Approach 2:
The patent introduces virtual reference pictures as an intermediary element between the decoded reference pictures and the current picture being encoded. These virtual reference pictures act as a mediator that bridges the gap, providing enhanced prediction material that more closely resembles the actual current picture content, thereby reducing residual data.
2Measurement precision
If more accurate reference pictures are generated using neural-network processing, then prediction accuracy is improved, but device complexity increases
Solution Approach 1:
The patent creates virtual reference pictures by copying and transforming existing decoded reference pictures through neural network processing. Instead of requiring completely new complex processing systems, the approach replicates and enhances existing reference materials, achieving improved accuracy while managing complexity through reuse of existing picture data.
3Loss of information
If virtual reference pictures are generated from decoded reference pictures, then residual data is reduced, but processing time increases
Solution Approach 1:
The virtual reference pictures are generated in advance during the decoding of reference pictures, before they are needed for the current picture encoding. This preliminary action allows the system to prepare enhanced prediction material without adding time pressure to the critical encoding path, reducing residual data efficiently.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of performing inter-picture prediction of a current picture of a video sequence is performed by at least one processor and includes determining whether the current picture uses a virtual reference picture, and based on the current picture being determined to use the virtual reference picture, generating, for the current picture, the virtual reference picture from neighboring reference pictures that are nearest neighbors to the virtual reference picture, storing the generated virtual reference picture in a decoded picture buffer, and adding the generated virtual reference picture to an active reference picture list. The method further includes performing the inter-picture prediction of the current picture, based on the active reference picture list to which the virtual reference picture is added.