Warping Engine Video Codec Prediction Modes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video codecs lack effective prediction modes for frames synthesized using computer graphics methods, leading to suboptimal compression efficiency, especially in GPU-rendered content, which results in higher bandwidth and storage requirements.
Innovation Solution
Incorporating a warping engine that generates a warped video frame based on additional rendering information, such as camera projection and Z-depth data, as a reference frame for compression, alongside a packetizer to separate and transmit main and auxiliary data streams, allowing for improved prediction and reduced data transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If traditional video prediction modes are used for GPU-rendered content, then compression is applied, but compression efficiency is suboptimal leading to higher bandwidth requirements
Solution Approach 1:
The system performs preliminary warping of the previous video frame using rendering information (camera projection matrix and Z-depth buffer) to create a warped reference frame before the compression process. This preliminary geometric transformation aligns the reference frame with the current frame's perspective, enabling more efficient compression by reducing the residual data that needs to be encoded.
Solution Approach 2:
The invention changes the parameter space by introducing warped reference frames generated from rendering information alongside traditional reference frames. This adds a new dimension to the prediction process, allowing the encoder to select from multiple reference frame types (original and warped) to optimize compression for GPU-rendered content with camera motion.
2Manufacturing precision
If more data is transmitted to maintain image quality, then image quality is preserved, but transmission time and latency increase
Solution Approach 1:
By pre-computing the warped reference frame using available rendering information at the encoder side, the system reduces the amount of data that needs to be transmitted to achieve the same image quality. The decoder reconstructs the current frame by combining the warped reference frame with transmitted residuals, maintaining image quality while reducing transmission time.
3Loss of energy
If traditional compression methods are used, then bandwidth is reduced, but the amount of data required for high-quality reconstruction remains high
Solution Approach 1:
The system segments the compression process into two parts: (1) transmission of a warped reference frame generated from rendering information, and (2) transmission of smaller residual data. This segmentation allows the decoder to reconstruct high-quality frames by combining these two components, reducing the total data volume compared to traditional methods that transmit only full-frame residuals.
Data Source
AI summary
A video frame compression system includes a rendering engine that provides a current video frame and current additional rendering information. Additionally, the video frame compression system includes a warping engine that generates a warped video frame, wherein the warped video frame is a transformation of a previous video frame that is based on the current additional rendering information. Further, the video frame compression system includes a video encoder that compresses the current video frame by using the warped video frame as a reference frame and separately compresses the current additional rendering information. Still further, the video frame compression system includes a packetizer that provides main and auxiliary data streams corresponding to the compressed current video frame and the compressed current additional rendering information, respectively. A video frame decompression system and methods of video frame compression and decompression are also provided.