Cloud Gaming Display Timing Control via Rendering Time Data

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Cloud gaming technologies face challenges in maintaining low latency and synchronizing image display intervals between cloud servers and terminals due to variable frame rates and varying processing loads, leading to inconsistent image rendering and display timing.

Innovation Solution

A display control apparatus with a reception section, display timing determination section, and display control section that receives image data with rendering time data, determines display timing based on latency and cumulative error, and controls image display accordingly, including the option to adjust data rates based on tolerable latency statistics.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Stability of the object's composition

If the terminal displays images at a constant frame rate, then the display operation is simple and stable, but the image display time interval cannot follow the variable image rendering time interval at the cloud server

Engineering Contradiction:

Improvedisplay frame rate stabilityVSAvoidadaptation to variable rendering intervals

Core Design Contradiction:

Stability of the object's compositionVSAdaptability or versatility

Solution Approach 1:

The patent applies dynamics by enabling the display terminal to switch between constant frame rate mode and variable frame rate mode. The display control section dynamically adjusts the display frame rate based on received rendering time data, allowing the system to adapt to varying rendering intervals while maintaining operational simplicity when conditions permit constant framing.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent changes the display frame rate parameter from a fixed constant value to a variable parameter that adjusts according to the rendering time interval. The display control section modifies the display timing parameter based on received rendering time data, enabling the system to match the cloud server's variable rendering pace when needed.

Inventive Principle:

Principle #35Parameter changes

2Ease of manufacture

If the terminal performs display operation in accordance with image reception, then the implementation is simple, but the image display time interval cannot properly follow the image rendering time interval due to varying latency

Engineering Contradiction:

Improveimplementation simplicityVSAvoiddisplay timing accuracy

Core Design Contradiction:

Ease of manufactureVSManufacturing precision

Solution Approach 1:

The patent implements feedback by having the display terminal receive rendering time data from the cloud server and use this information to adjust its display timing. The display control section continuously monitors the relationship between image reception timing and rendering time interval, adjusting display operations based on this feedback to achieve accurate synchronization.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent applies preliminary action by having the cloud server transmit rendering time data along with image data before the display operation occurs. This allows the display terminal to pre-calculate the appropriate display timing based on the received rendering time interval, ensuring accurate synchronization when the image is actually displayed.

Inventive Principle:

Principle #10Preliminary action

3Adaptability or versatility

If variable frame rate display is implemented, then the image display time interval can follow the rendering time interval, but the display device complexity increases

Engineering Contradiction:

Improvevariable frame rate supportVSAvoiddisplay control complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent uses an intermediary approach by introducing a display control section that acts as a mediator between the image reception unit and the display unit. This control section receives rendering time data, calculates the appropriate display timing, and coordinates the display operation, thereby managing the complexity centrally rather than distributing it throughout the system.

Inventive Principle:

Principle #24Intermediary (Mediator)

4Ease of operation

If images are displayed without considering latency and cumulative error, then the display operation is simple, but the timing synchronization between cloud server rendering and terminal display deteriorates

Engineering Contradiction:

Improvedisplay operation simplicityVSAvoidtiming synchronization accuracy

Core Design Contradiction:

Ease of operationVSMeasurement precision

Solution Approach 1:

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11984059B2Display control apparatus, transmission apparatus, display control method, and program

Publication Date: 2024.05.14 SONY INTERACTIVE ENTERTAINMENT LLC

AI summary

A display control apparatus, a transmission apparatus, a display control method, and a program that enable an image display time interval at the display control apparatus, which is connected to the transmission apparatus through a network, to properly follow an image rendering time interval at the transmission apparatus. A data reception section sequentially receives image data indicating an image associated with rendering time data capable of determining an image rendering time for the image in a cloud server. A display timing determination section determines, based on the rendering time data, a display timing of the image indicated by the image data. A display control section controls display of the image in accordance with the determined display timing.