Video Coding System External Reference Picture Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding technologies in cloud gaming face inefficiencies in compressing high-quality graphics due to limitations in existing layered coding frameworks, particularly in inter-layer prediction and external reference picture usage, leading to suboptimal compression efficiency when dealing with partially rendered images on client devices.
Innovation Solution
The proposed solution involves a video coding system that utilizes an enriched set of reference pictures and external reference pictures to enable temporal prediction, allowing for pure motion-compensated temporal prediction equivalent to cases where the base layer is not used, thereby increasing compression efficiency by using differential coding and additional reference pictures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional video coding technologies are used for cloud gaming, then implementation is simpler, but compression efficiency is insufficient for high-quality graphics
Solution Approach 1:
The patent segments the video coding process into multiple layers (base layer and enhancement layer) with different quality levels. The base layer contains lower-quality reference pictures while the enhancement layer contains higher-quality current pictures, allowing differential coding of only the quality difference portion. This segmentation enables efficient compression of high-quality graphics by only transmitting the incremental information needed to enhance the base layer.
Solution Approach 2:
The patent introduces a new dimension to the reference picture buffer by incorporating external reference pictures from graphics renderers alongside traditional decoded video pictures. This creates a multi-dimensional reference structure where pictures can be selected from different sources (decoded video vs. external renderer) and different quality levels, enabling more flexible and efficient prediction modes for compressing cloud gaming content.
2Productivity
If existing layered coding frameworks are used, then implementation is more straightforward, but inter-layer prediction is suboptimal
Solution Approach 1:
The patent creates a universal reference picture buffer that can store and manage multiple types of reference pictures (base layer pictures, enhancement layer pictures, external renderer pictures) in a unified structure. This multi-functional buffer supports various prediction modes (inter-layer prediction, temporal prediction, differential coding) and allows the system to adaptively select the most efficient prediction approach for each picture or block, improving compression efficiency while managing complexity through standardized interfaces.
3Productivity
If external reference pictures are incorporated, then compression efficiency improves, but management of reference picture buffer becomes more complex
Solution Approach 1:
The patent implements a dynamic reference picture buffer management system that can adaptively adjust the contents, size, and organization of the buffer based on current coding conditions. The buffer dynamically adds external reference pictures from graphics renderers when beneficial, removes less useful pictures, and adjusts the weighting and prioritization of different reference sources. This dynamic management enables efficient bitrate reduction through optimal picture selection while handling the increased complexity through adaptive algorithms that respond to changing video content and quality requirements.
Data Source
AI summary
A video coding system for an image of a video representative of a virtual environment comprises an encoder and a decoder that perform a reconstruction of a differential picture based on a decoded picture buffer, wherein the decoded picture buffer comprises a least a picture based on a reference differential picture and on a second picture generated by a three-dimensional renderer based on the information representative of the virtual environment. An encoding method and a decoding method are described and comprise adding the reconstructed differential picture is to the second picture.