Generative Frame Interpolation for Dropped Video Streams

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Video streaming experiences are degraded due to network conditions or device malfunctions that cause dropped video frames, leading to discontinuities.

Innovation Solution

Utilizing a generative machine-learning model to create replacement frames based on contextual information from the video stream, including metadata and data from surrounding frames, to fill or eliminate these discontinuities.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If network conditions are unstable or device malfunctions occur, then video streaming can be transmitted, but dropped video frames cause discontinuities that degrade viewing experience

Engineering Contradiction:

Improvevideo stream continuityVSAvoiddropped frames causing discontinuities

Core Design Contradiction:

ReliabilityVSObject-affected harmful factors

Solution Approach 1:

A generative machine-learning model is introduced as an intermediary component between the video stream and the user. This model receives the video stream, identifies dropped frames, and generates replacement frames that fill the discontinuities, thereby mediating the harmful effect of frame drops without blocking the underlying video transmission

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system creates copies of missing video frames by generating synthetic frame data that replicates the content of dropped frames. The generative model analyzes surrounding frames and metadata to produce accurate replacements, effectively copying the missing content to maintain stream continuity

Inventive Principle:

Principle #26Copying

2Reliability

If replacement frames are generated using generative machine-learning, then viewing experience is enhanced, but computational resources and processing time are consumed

Engineering Contradiction:

Improveviewing experience qualityVSAvoidcomputational resources

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

Instead of processing the entire video stream continuously, the system applies machine-learning processing only to specific segments where frame drops are detected. The generative model is activated selectively to generate replacement frames only when discontinuities occur, reducing overall computational resource consumption while maintaining quality where needed

Inventive Principle:

Principle #16Partial or excessive action

3Reliability

If frame replacement is performed in real-time, then playback continuity is maintained, but processing delay increases

Engineering Contradiction:

Improveplayback continuityVSAvoidprocessing delay

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary analysis of the video stream to identify dropped frames and their positions before generating replacements. By pre-processing the stream to detect discontinuities and prepare replacement frames in advance, the system minimizes processing delay during actual playback, maintaining smooth continuity

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20260052224A1Using generative machine-learning to interpolate dropped frames

Publication Date: 2026.02.19 ROKU INC

AI summary

Aspects of the disclosed technology provide solutions for improving video streams by generating dropped video frames. An example process can include steps for receiving a set of video frames, identifying a discontinuity in the set of frames, generating one or more replacement frames associated with the discontinuity, and providing the one or more replacement frames to a user. Systems and machine-readable media are also provided.