Client Image Frame Decoding With Prediction for Network Game Continuity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Video games accessed over computer networks often experience frame loss, leading to disruptions in the display of image frames, which can affect the frame rate and gameplay experience.
Innovation Solution
Client devices predict and generate estimated frames using motion compensation and, if necessary, machine learning, to maintain the frame rate by displaying estimated frames when encoded frames are not received.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If image frames are transmitted over a computer network, then video games can be accessed remotely, but frame loss occurs during transmission causing disruptions in display
Solution Approach 1:
The system performs preliminary actions by predicting and generating estimated frames before the actual encoded frames are received. The prediction process uses motion compensation and machine learning to create placeholder frames that maintain display continuity, effectively preparing for potential frame loss in advance
Solution Approach 2:
The system creates copies or approximations of the expected frames using prediction algorithms. Instead of waiting for the original encoded frames, the client generates duplicate visual content through motion compensation techniques that replicate the anticipated frame content, ensuring display continuity when originals are lost
2Reliability
If the system waits for encoded frames to be received before displaying them, then transmission accuracy is maintained, but frame rate and gameplay smoothness deteriorate due to frame loss
Solution Approach 1:
The system performs preliminary frame generation using prediction algorithms before the actual frames are received or before display is needed. Motion compensation predicts future frame content based on historical data, creating estimated frames in advance that can be immediately displayed without waiting for network transmission
Solution Approach 2:
The prediction system acts as an intermediary between the network transmission and the display system. When encoded frames are lost in transmission, the predicted frames serve as intermediate substitutes that bridge the gap, allowing the display system to continue operating at full frame rate without directly receiving the original frames
3Productivity
If motion compensation and machine learning are used to predict frames, then frame rate is maintained during transmission losses, but device complexity increases
Solution Approach 1:
The client device performs self-service by generating its own predicted frames using local machine learning models and motion compensation algorithms. Instead of relying on the server to provide all necessary frame data, the client independently creates missing frames using its own computational resources and historical data
Data Source
AI summary
A method for decoding image frames at a client is described. The method includes generating an estimated image frame after receiving an encoded image frame of a stream of encoded image frames. The method further includes decoding the encoded image frame when the encoded image frame is received at a decode interval set for a frame rate of presentation. The method includes using the estimated image frame when a corresponding encoded image frame fails to arrive within the stream for presentation at the frame rate.