ML Model Reconstructs Missing Game Frame Slices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In video streaming, particularly computer game streaming, frames or frame portions may be lost entirely or received with low quality, affecting the presentation of the frame and potentially other frames that reference the missing portions.
Innovation Solution
A machine learning (ML) model is trained on ground truth gameplay video recordings to reconstruct missing or low-quality frame portions, enhancing their quality and ensuring accurate frame presentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If video frames are transmitted over a network, then bandwidth utilization is improved, but frame loss and quality degradation occur
Solution Approach 1:
The system performs preliminary actions by training the ML model on ground truth gameplay recordings before actual streaming. The model learns to reconstruct frames in advance, enabling it to compensate for network losses during transmission without requiring retransmission or additional bandwidth during live streaming.
Solution Approach 2:
The ML model acts as an intermediary between the transmitted video data and the final displayed frame. When frame portions are lost or degraded during network transmission, the model reconstructs the missing information by predicting what the original content should be, based on its training on ground truth gameplay data.
2Productivity
If frame portions are lost or received with low quality, then network transmission efficiency is improved, but presentation quality of current and subsequent frames deteriorates
Solution Approach 1:
The system converts the harmful effect of frame loss into a beneficial outcome by using the ML model to reconstruct missing frame portions. Instead of requiring perfect transmission, the model learns from ground truth data to predict and restore lost content, turning network imperfections into an opportunity for intelligent reconstruction that maintains or even improves presentation quality.
Solution Approach 2:
The system changes the parameter of frame quality from a fixed transmission-dependent value to a dynamically reconstructed value. The ML model adjusts the quality parameters of reconstructed frames based on the specific loss patterns and content characteristics, allowing high-quality presentation even when transmission conditions are poor.
3Measurement precision
If ML model is trained on ground truth gameplay recordings, then reconstruction accuracy is improved, but training time and computational resources increase
Solution Approach 1:
The system performs the computationally intensive training operation in advance, before actual streaming begins. By training the ML model on ground truth gameplay recordings beforehand, the system accumulates reconstruction accuracy without impacting real-time streaming performance. The trained model can then be deployed for efficient inference during live gameplay transmission.
Data Source
AI summary
Techniques are described for over-training a ML model on multiple gameplay videos of individual scenes of a computer game to better configure the model to reconstruct or enhance portions of the computer game at a receiver as the computer game is received over a streamlining network. Reconstruction of individual missing slices of a frame is contemplated such that a frame missing a slice need not be entirely discarded.