Neural Shading of Reflective Surfaces for Fast AR Rendering

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing neural rendering techniques struggle to produce realistic representations of reflective (glossy) surfaces in augmented reality objects, particularly failing to model specular reflections accurately.

Innovation Solution

Utilize neural rendering techniques combined with physically based rendering to create explicit 3D assets, employing image-to-image neural networks trained on extracted product meshes to generate photorealistic views of reflective surfaces.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If existing neural rendering techniques are used, then processing speed is improved, but the realism of reflective surfaces deteriorates

Engineering Contradiction:
Improveprocessing speedVSAvoidrealism of reflective surfaces
Core Design Contradiction:
SpeedVSManufacturing precision

Solution Approach 1:

The patent applies preliminary action by pre-training neural networks on extensive datasets of reflective surfaces and pre-computing rendering results for various lighting conditions. This allows the system to quickly generate realistic reflections during actual use without requiring real-time complex calculations, thus maintaining both speed and realism.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent uses copying by creating synthetic training data that replicates real-world reflective surfaces and lighting conditions. Neural networks are trained on these synthetic copies to learn accurate reflection patterns, which are then applied to render realistic reflective surfaces efficiently without needing to process every physical detail in real-time.

Inventive Principle:
Principle #26Copying

2Manufacturing precision

If physically based rendering is used to improve realism, then rendering quality improves, but computational complexity increases

Engineering Contradiction:
Improverendering qualityVSAvoidcomputational complexity
Core Design Contradiction:
Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent replaces complex mechanical rendering calculations with neural network-based approaches. Instead of performing computationally intensive ray tracing and physics-based calculations in real-time, the system uses pre-trained neural networks that have encoded the complexities of light interaction with reflective surfaces, substituting complex computations with simpler neural inference operations.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent applies parameter changes by transforming the rendering problem from a physics-based calculation approach to a data-driven neural network approach. The system changes the fundamental parameters of how rendering is achieved, using learned patterns from training data rather than explicit physical models, thereby reducing computational complexity while maintaining quality.

Inventive Principle:
Principle #35Parameter changes

3Manufacturing precision

If neural networks are trained on extensive data to improve rendering accuracy, then rendering accuracy improves, but training time and resources increase

Engineering Contradiction:
Improverendering accuracyVSAvoidtraining time
Core Design Contradiction:
Manufacturing precisionVSLoss of time

Solution Approach 1:

The patent applies preliminary action by performing all intensive training work beforehand, creating pre-trained neural network models that can be quickly deployed. The extensive data processing and model training are completed in advance, allowing the actual rendering task to proceed rapidly without requiring time-consuming training during execution.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent uses partial or excessive action by training neural networks on comprehensive datasets that may exceed the minimum required for achieving good rendering accuracy. This excessive training on diverse lighting conditions and surface properties ensures the models generalize well to new situations, providing robust accuracy that justifies the initial time investment.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentUS20250252652A1Neural shading of reflective surfaces
Publication Date: 2025.08.07 SNAP INC
  • US20250252652A1 patent drawing
  • US20250252652A1 patent drawing
  • US20250252652A1 patent drawing

AI summary

The subject technology receives an object mesh, information related to a viewpoint for rendering an image of an object having a reflective surface, and a set of maps. The subject technology generates a rasterized RGB (Red Green Blue) image based on the object mesh, the viewpoint, and the set of maps. The subject technology generates, using a neural network model, an output image of the object with the reflective surface based at least in part on the rasterized RGB image and the viewpoint. The subject technology provides for display the output image of the object with the reflective surface on a display of a computer client device.