Client-Side Frame Interpolation for Missing Cloud Gaming Streams

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In cloud gaming, network latency can cause frames to be missing, disrupting the natural game screen, and existing technologies increase network and server load by sending speculative images.

Innovation Solution

A client terminal equipped with a generative Neural Network model interpolates missing images using optical flow and depth information from buffered frames, generating an estimated image to maintain a natural game screen without increasing server load.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If speculative images are sent in advance from server to client terminal, then the natural game screen is maintained, but the network load and server load increase

Engineering Contradiction:

Improvenatural game screenVSAvoidnetwork load

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

Instead of having the server generate and send speculative images in advance, the invention inverts the approach by having the client terminal generate interpolated images locally using a generative AI model. This shifts the computational burden from server to client, reducing network traffic while maintaining screen naturalness.

Inventive Principle:

Principle #13The other way round (Inversion)

Solution Approach 2:

The invention introduces a generative AI model as an intermediary at the client terminal that creates missing frames by interpolating between received frames. This intermediary component generates realistic intermediate images without requiring actual server computation or additional network transmission.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If speculative images are sent in advance from server to client terminal, then the natural game screen is maintained, but the server load increases

Engineering Contradiction:

Improvenatural game screenVSAvoidserver load

Core Design Contradiction:

ReliabilityVSPower

Solution Approach 1:

The invention inverts the computational responsibility from server to client. The client terminal's generative AI model performs the image interpolation locally, eliminating the need for server-side speculative rendering and reducing server computational load.

Inventive Principle:

Principle #13The other way round (Inversion)

Solution Approach 2:

The client terminal serves itself by generating missing frames through its local generative AI model, rather than relying on the server to provide speculative images. This self-service approach reduces server burden while maintaining image quality.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS20260007962A1Program, client terminal, game system, and processing method

Publication Date: 2026.01.08 CYGAMES INC

AI summary

A client terminal may include an estimation model for interpolating a missing streaming image. The estimation model may estimate an optical flow and depth information from streaming images I(0) to I(t) displayed before a missing streaming image I(t+1) and generate a streaming image I′(t+1) on the basis of the estimation result. The client terminal may display the generated streaming image I′(t+1) instead of the missing streaming image I(t+1).