ROI Video Decoding with Multiple Encoders for Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Network bandwidth limitations and regulatory constraints can lead to latency issues in streaming computer game videos, where near-instantaneous reaction to gamer inputs is critical, making video quality a secondary concern.
Innovation Solution
The system employs multiple encoders to prioritize the transmission of a first portion of the video, which includes a region of interest (ROI), over a network, while using secondary encoders for other portions. Decoders are configured to present the ROI immediately, even if other parts of the video are not available within a latency period, by either reusing previous frames or obfuscating surrounding video portions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If video is transmitted over network with bandwidth limitations, then network channel capacity is preserved, but transmission latency increases
Solution Approach 1:
The video is segmented into multiple portions with different priority levels. High-priority portions (containing critical game actions) are transmitted first through the network, while lower-priority portions are transmitted subsequently or reconstructed. This segmentation allows the system to transmit essential visual information with minimal latency while conserving network bandwidth by deferring or omitting less critical data.
Solution Approach 2:
Different portions of the video are transmitted with different quality levels and priorities. Regions containing critical game actions (such as weapon firing, character movement) are transmitted with high priority and quality, while other regions are transmitted with lower priority or reconstructed. This local quality differentiation ensures that bandwidth is allocated to the most important visual information, minimizing latency for critical game moments.
2Loss of time
If multiple encoders are used to prioritize video portions, then transmission latency is reduced, but device complexity increases
Solution Approach 1:
The encoding function is segmented across multiple encoder components, where each encoder handles specific priority levels or video portions. This allows parallel processing of different video regions with different compression settings, reducing overall encoding time and latency while distributing computational complexity across multiple specialized units rather than requiring one complex universal encoder.
3Loss of time
If ROI portions are transmitted first, then latency for critical actions is minimized, but video completeness is reduced
Solution Approach 1:
The system performs preliminary transmission of high-priority video portions containing critical game actions before transmitting or reconstructing the remaining portions. This preliminary action ensures that the most time-sensitive visual information reaches the display device first, minimizing latency for critical game moments, while lower-priority portions are handled subsequently or through reconstruction algorithms.
Solution Approach 2:
When complete video frames are not available within the required latency period, the system creates reconstructed copies of missing video portions using available data and interpolation techniques. These reconstructed portions are then combined with the actually transmitted high-priority portions to form complete displayable frames, maintaining video completeness while preserving low latency for critical content.
Data Source
AI summary
Techniques are described for reducing latency in computer game network streaming by using plural encoders and decoders, with one encoder-decoder pair being used for regions of interest (ROI) in the video and being given priority in transmitting and rendering over background video that is processed by another encoder/decoder pair.