AR Object Rendering with Neural Network Blur and Noise Matching

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing augmented reality (AR) systems lack realistic camera imperfections such as blur and noise, leading to unnatural renderings of AR objects, which can 'jump out' of the frame and detract from the photorealism of the experience.

Innovation Solution

An AR object rendering system that estimates blur and noise in images using end-to-end neural network pipelines, allowing AR objects to be rendered with matching imperfections, thereby enhancing their integration into the background environment.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If AR objects are rendered with high sharpness and clarity, then the rendering quality of AR objects is improved, but the photorealism decreases because the objects appear unnatural and jump out of the frame

Engineering Contradiction:
Improverendering sharpnessVSAvoidphotorealism
Core Design Contradiction:
Manufacturing precisionVSReliability

Solution Approach 1:

The patent applies different quality characteristics to different parts of the image: AR objects receive high-quality rendering with sharpness and clarity, while the background receives lower-quality rendering with blur and noise. This local differentiation allows AR objects to maintain rendering precision while the overall scene achieves photorealism through selective application of camera imperfections.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

Instead of adding imperfections to already-rendered AR objects, the patent inverts the approach by first analyzing the background image to extract blur and noise characteristics, then using these characteristics to guide the rendering process. This ensures that AR objects are rendered with appropriate imperfections from the outset, naturally blending with the background while maintaining their visual prominence.

Inventive Principle:
Principle #13The other way round (Inversion)

2Reliability

If AR objects are rendered with camera imperfections like blur and noise, then the photorealism is improved, but the rendering complexity increases due to the need for neural network pipelines

Engineering Contradiction:
ImprovephotorealismVSAvoidrendering system complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent creates a digital copy or model of the camera's imperfection characteristics by analyzing the background image to extract blur and noise parameters. This copied information is then applied to AR object rendering, avoiding the need for complex physical camera simulations while achieving realistic results through parameter transfer from the background analysis.

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The patent replaces traditional mechanical or optical methods of simulating camera imperfections with neural network-based image analysis. Instead of using complex optical models or physical simulations, the system uses machine learning pipelines to automatically extract and apply blur and noise characteristics, significantly reducing computational complexity while maintaining photorealism.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

3Speed

If the rendering system processes images in real-time, then the AR experience responsiveness is improved, but the image quality analysis precision decreases due to processing time constraints

Engineering Contradiction:
Improveprocessing speedVSAvoidblur and noise estimation accuracy
Core Design Contradiction:
SpeedVSMeasurement precision

Solution Approach 1:

The patent performs preliminary analysis of the background image to extract blur and noise characteristics before rendering AR objects. By pre-processing the background to identify camera imperfection patterns, the system prepares the necessary parameters in advance, enabling fast rendering without compromising the accuracy of blur and noise estimation. This preliminary action ensures both real-time performance and precise quality analysis.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS12240515B2Augmented reality object rendering based on camera quality
Publication Date: 2025.03.04 SNAP INC
  • US12240515B2 patent drawing
  • US12240515B2 patent drawing
  • US12240515B2 patent drawing

AI summary

Systems and embodiments herein describe an augmented reality (AR) object rendering system. The AR object rendering system receives an image, generates a set of noise parameters and a set of blur parameters for the image using a neural network trained on a paired dataset of images, identifies an AR object associated with the image, modifies the AR object using the set of noise parameters and the set of blur parameters, displays the modified augmented reality object within the image.