Frame Token Timing Compensation for Cloud Game Input Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Video game applications face challenges with end-to-end latency and synchronization of user input with frame presentation, leading to difficulties in correlating game events with player inputs, especially in cloud-based streaming systems where latency can be significant and variable.
Innovation Solution
The implementation of frame tokens, which are generated by a server and transmitted to a client, include timing information associated with frame processing and user inputs, allowing the application to compensate for latency and correlate inputs with the correct frame, even if there is a delay between frame generation and input reception.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If video game application program code is stored locally on the player's device, then the player can access and play games without network dependency, but the player must maintain large storage capacity and experience inconvenient delivery requiring physical disks or lengthy downloads
Solution Approach 1:
The patent extracts the game application program code from the player's local device and relocates it to a remote server. The server stores and executes the game code, while the player's device only needs to run a client application that communicates with the server. This extraction eliminates the need for large local storage capacity while maintaining easy game access through network connectivity.
Solution Approach 2:
The patent introduces a server as an intermediary between the game distributor and the player's device. The server acts as a middleman that stores the game code, manages game execution, and communicates with the player's client application. This intermediary approach allows the player to access games without storing large amounts of data locally, as the server handles all code storage and distribution operations.
2Productivity
If video game program code is delivered via physical disk or download, then the game can be distributed to players, but the delivery process takes up to several hours for large games and requires purchasing or downloading
Solution Approach 1:
The patent implements preliminary action by pre-loading and caching game application program code on the remote server before the player needs to access it. The server maintains a library of game codes ready for immediate execution. When a player wants to play a game, the client application simply requests the game from the server, which can immediately provide it without requiring the player to wait for download or physical media insertion. This preliminary preparation eliminates acquisition delays.
3Quantity of substance
If cloud-based streaming is used to deliver games, then storage requirements are reduced and delivery is faster, but end-to-end latency and variable network delays make it difficult to correlate game events with player inputs
Solution Approach 1:
The patent implements feedback mechanisms where the server continuously monitors network latency and transmission timing, and this information is fed back to the client application. The server measures the end-to-end delay for each frame transmission and communicates this timing data back to the client. The client uses this feedback to compensate for latency by adjusting the timing of input processing and frame rendering, thereby maintaining accurate correlation between player inputs and game events despite network variability.
Solution Approach 2:
The patent applies parameter changes by dynamically adjusting timing parameters on both the server and client sides based on measured network conditions. The server modifies frame transmission timing and the client adjusts input processing timing based on real-time latency measurements. By changing these temporal parameters adaptively, the system compensates for network variability and maintains precise synchronization between player inputs and game frame presentation, resolving the correlation accuracy problem.
Data Source
AI summary
A server executing an application generates a frame token for a frame that is rendered for the application. One or more first metric messages are provided to the application in response to at least one first operation performed by the server on the frame. The first metric messages include the frame token and information indicating timing of the at least one first operation. The encoded information representing the frame token and the frame is transmitted from the server towards a client. One or more second metric messages are provided to the application in response to one or more second operations performed by the client on the frame. The one or more second metric messages include the frame token and information indicating timing of the second operations. A state of the application is modified based on the first and second metric messages.