Generation Super Sampling Using Autoencoders for Fixed Frame Rates

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing computer graphics systems experience variable frame rates, leading to noticeable latency and user experience degradation due to 'hitching' and screen tearing, especially when transitioning between processing intensive and less intensive tasks.

Innovation Solution

Implementing an autoencoder neural network to generate synthetic frames using machine learning algorithms, allowing for fixed frame rates by predicting subsequent frames based on previous frames and user inputs, independent of the rendering speed of real frames.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If synthetic frames are inserted between real frames to increase frame rate, then frame rate is improved, but latency increases and variable frame rate causes hitching

Engineering Contradiction:
Improveframe rateVSAvoidlatency
Core Design Contradiction:
ProductivityVSLoss of time

Solution Approach 1:

The system performs preliminary actions by pre-rendering multiple real frames in advance and pre-training the autoencoder model beforehand. The autoencoder is trained offline on large datasets of image streams, so during runtime it can rapidly generate synthetic frames without requiring real-time complex processing. This preliminary preparation enables the system to maintain high frame rates with reduced latency during actual operation.

Inventive Principle:
Principle #10Preliminary action

2Productivity

If multiple real frames are rendered to create synthetic frames, then higher frame rates are achieved, but processing time increases causing variable frame rate

Engineering Contradiction:
Improveframe rateVSAvoidprocessing time
Core Design Contradiction:
ProductivityVSLoss of time

Solution Approach 1:

The patent replaces the mechanical rendering system with an intelligent prediction system. Instead of relying solely on the GPU to render every frame mechanically, the autoencoder neural network substitutes this mechanical process by predicting and generating synthetic frames intelligently. The autoencoder learns the temporal patterns and motion dynamics from training data, enabling it to generate realistic intermediate frames without requiring the GPU to perform full rendering operations for each frame, thus reducing processing time while maintaining frame rate.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

3Productivity

If frame rate is increased through synthetic frame generation, then user experience improves, but visual artifacts like tearing and hitching occur

Engineering Contradiction:
Improveframe rateVSAvoidvisual quality consistency
Core Design Contradiction:
ProductivityVSReliability

Solution Approach 1:

The system implements feedback mechanisms by comparing the generated synthetic frames with the actual rendered frames and adjusting the generation process accordingly. The autoencoder is trained using feedback from reconstruction errors and motion prediction accuracy, continuously refining its ability to generate visually consistent frames. This feedback loop ensures that synthetic frames blend seamlessly with real frames, reducing visual artifacts like tearing and hitching while maintaining high frame rates.

Inventive Principle:
Principle #23Feedback

Data Source

PatentUS20250209568A1Generation super sampling
Publication Date: 2025.06.26 SONY INTERACTIVE ENTERTAINMENT LLC
  • US20250209568A1 patent drawing
  • US20250209568A1 patent drawing
  • US20250209568A1 patent drawing

AI summary

Frame generation super sampling may include generating an image frame embedding from a first image frame in an image stream and predicting a synthetic second image frame in the image stream using the image frame embedding of the first image in the image stream. The synthetic second image is displayed frame after the first image frame in the image stream.