Multi-Encoder Video Compression Context Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current client-server application models, particularly in remote gaming, face challenges in reducing end-to-end latency due to limitations in video compression and context switching mechanisms, which affect the quality and efficiency of video frame transmission.
Innovation Solution
The proposed solution involves an encoding apparatus and method that uses two encoders to generate predicted and fallback video streams based on user behavior, with context switching mechanisms to adapt encoding contexts on a frame-by-frame basis, enabling efficient video compression and reducing latency by selecting the appropriate video frames for transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If lossy H.264 or H.265 compression is used for video transmission, then bandwidth efficiency is improved, but video quality and compression accuracy deteriorate
Solution Approach 1:
The system performs preliminary actions by generating multiple predicted video frames in advance using different prediction methods (motion estimation, depth buffering, ray tracing) before the actual transmission occurs. These predicted frames are prepared and stored ready for selection, allowing the system to choose the best quality frame without real-time compression delays.
Solution Approach 2:
The system creates multiple copies of predicted video frames using different rendering approaches (motion-compensated prediction, depth-based rendering, ray tracing). By maintaining multiple copies with different quality characteristics, the system can select the most appropriate copy for transmission, balancing quality and bandwidth efficiency.
2Adaptability or versatility
If context switching mechanisms are used to adapt encoding contexts, then video transmission adaptability is improved, but system complexity increases
Solution Approach 1:
The system segments the video rendering process into distinct components: motion estimation module, depth buffering module, ray tracing module, and selection module. Each segment handles a specific prediction task independently, allowing flexible combination and selection without requiring complex integrated processing.
Solution Approach 2:
The system implements dynamic context switching where the encoding context can be changed on a frame-by-frame basis depending on the predicted user behavior and scene characteristics. The selection of which prediction method to use is dynamically adjusted based on real-time conditions, providing adaptability without permanent structural complexity.
3Reliability
If multiple encoders generate predicted and fallback video streams, then video transmission reliability is improved, but processing time increases
Solution Approach 1:
Multiple predicted video frames are generated in advance using parallel prediction methods before transmission is needed. The motion estimation, depth buffering, and ray tracing processes prepare multiple candidate frames ahead of time, so that when transmission occurs, the selection can be made quickly without real-time processing delays.
Solution Approach 2:
The system uses computationally less intensive prediction methods (motion estimation, depth buffering) to generate multiple candidate frames quickly, accepting that these are temporary approximations. The more accurate but expensive ray tracing is used selectively only when needed, rather than for every frame, reducing overall processing time while maintaining reliability.
Data Source
AI summary
Examples relate to an encoding apparatus, encoding device, encoding method, computer program and to a corresponding computer system. An encoding apparatus comprises processing circuitry configured to encode at least two video streams to perform video compression encoding using one or more encoders, wherein the one or more encoders are each configured to deter-mine an encoding context based on the video stream processed by the respective encoder, the encoding context comprising at least one or more reference frames determined by the respective encoder. The processing circuitry is configured to store at least one of the encoding contexts determined by the one or more encoders in a shared memory portion of the memory circuitry that is accessible to the one or more encoders. The processing circuitry is configured to proceed, using the one or more encoders, with encoding the at least two video streams based on an encoding context stored in the shared memory portion.