Gameplay Video Encoding Using Game Data for Low-Latency Quality Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video game streaming technologies face challenges in maintaining low latency and high visual quality due to the real-time generation of gameplay video, making it difficult to apply complexity estimation methods that increase latency.
Innovation Solution
Estimate the complexity of image frames based on game data such as audio, haptic, and text information using predefined algorithms or machine learning models, and adjust encoding parameters accordingly to optimize video quality and reduce bandwidth requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If complexity estimation methods are applied to gameplay video encoding, then visual quality is improved, but latency increases
Solution Approach 1:
The system performs complexity estimation on game data (audio, haptics, text) before the video is actually rendered and encoded. By predicting the complexity of future frames in advance based on game state information, the encoder can prepare appropriate encoding parameters without waiting for the actual video frames, thus avoiding latency while maintaining quality optimization
Solution Approach 2:
The patent introduces game data (audio, haptics, text) as an intermediary to estimate video complexity. Instead of directly analyzing video frames to determine complexity, the system uses these alternative game data sources as proxies to predict visual complexity, enabling quality optimization without the latency of actual frame analysis
2Manufacturing precision
If encoding parameters are adjusted to maintain high visual quality, then bandwidth requirements increase
Solution Approach 1:
The system dynamically adjusts encoding parameters based on the estimated complexity of each frame. When complexity is low, more aggressive compression is applied; when complexity is high, better quality settings are used. This dynamic adaptation maintains high visual quality where needed while reducing bandwidth consumption during less demanding scenes
Solution Approach 2:
The patent changes encoding parameters (such as bitrate, resolution, compression level) based on the estimated complexity derived from game data. By varying these parameters according to actual content complexity rather than using fixed settings, the system optimizes the balance between visual quality and bandwidth utilization
3Loss of time
If real-time encoding is performed without complexity estimation, then latency is reduced, but visual quality deteriorates
Solution Approach 1:
By estimating complexity in advance from game data before encoding occurs, the system prepares optimal encoding parameters beforehand. This preliminary action allows the actual encoding process to proceed quickly without iterative quality adjustments, maintaining low latency while ensuring high visual quality through pre-planned parameter selection
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
An encoding system configured to encode video of a game being executed, the video being encoded for transmission to a client device operated by a player of the game, the system comprising a game execution unit configured to execute the game, wherein executing the game comprises rendering a plurality of image frames for display to the player, a game information obtaining unit configured to obtain information about the game, including audio and/or text information associated with an image frame being rendered, a complexity estimation unit configured to estimate a spatial and/or temporal complexity of the image frame being rendered in dependence upon the obtained information, a parameter selection unit configured to select one or more encoding parameters in dependence upon the estimated spatial and/or temporal complexity, and an encoding unit configured to encode the video of the game being executed using the selected encoding parameters, the encoded video comprising the plurality of image frames for display to the player.