Neural Network Lossy Renderer for Real-Time High-Resolution Gaming

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current rendering technologies for games and simulations require either increasing the number of triangles or shader complexity to enhance realism, which results in performance degradation, especially on devices with limited compute and memory capacity, and cannot effectively compress information without sacrificing quality.

Innovation Solution

A neural network-based lossy rendering system that generates high-resolution images from low-resolution data maps by training a neural network to associate low-resolution inputs with high-resolution outputs, allowing for real-time high-definition image generation on low-end devices with minimal processing power.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If the number of triangles or shader complexity is increased to enhance realism, then image quality is improved, but device performance deteriorates

Engineering Contradiction:

Improveimage qualityVSAvoiddevice performance

Core Design Contradiction:

Manufacturing precisionVSProductivity

Solution Approach 1:

The patent replaces traditional geometric rendering (triangles and shaders) with a neural network-based approach. Instead of using complex geometric models and shader programs to generate high-quality images, the system trains a neural network to learn the mapping from low-resolution to high-resolution images, substituting the mechanical rendering process with an intelligent model that can be evaluated efficiently on mobile devices

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The neural network is trained in advance using paired low-resolution and high-resolution images. This preliminary training phase creates a pre-computed model that captures the relationship between different resolution levels. During actual rendering, the pre-trained network can quickly generate high-resolution outputs from low-resolution inputs without requiring complex real-time computations

Inventive Principle:

Principle #10Preliminary action

2Productivity

If information compression is applied to improve performance, then processing speed is improved, but image quality deteriorates

Engineering Contradiction:

Improveprocessing speedVSAvoidimage quality

Core Design Contradiction:

ProductivityVSManufacturing precision

Solution Approach 1:

The patent introduces a neural network as an intermediary between low-resolution input and high-resolution output. Instead of directly compressing information, the network learns to reconstruct high-frequency details and textures that would be lost in traditional compression, acting as a mediator that recovers quality information from compressed representations

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS11232623B2Method and system for creating a neural net based lossy renderer

Publication Date: 2022.01.25 UNITY TECH APS

AI summary

A method of generating a high-resolution image frame for a state of a video game within a 2D or 3D environment is disclosed. A low-resolution data map of a virtual camera frustum view of the 2D or 3D environment for the state is determined. The data map is of a data type. A high-resolution output data map of the data type is generated from the low-resolution data map. The generating of the high-resolution output data map includes training a neural network. The training includes associating a low-resolution data map of the data type with a high-resolution data map of the data type within the 2D or 3D environment. A high-resolution image of the frustum view is generated from the high-resolution output data map. The generated high-resolution image is displayed on a display device.