Augmented Reality Image Augmentation via Machine Learning

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing augmented reality content systems rely on user input for generating image augmentations, limiting creativity and realism in augmented reality experiences.

Innovation Solution

A custom augmentation system utilizing machine learning models to generate image augmentation decisions without user input, leveraging segmentation, classification, object detection, or saliency models to provide contextual and realistic modifications to images within augmented reality content items.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If user input is required to generate image augmentations, then user control and customization are improved, but system complexity and user burden increase

Engineering Contradiction:
Improveuser controlVSAvoidsystem complexity
Core Design Contradiction:
Ease of operationVSDevice complexity

Solution Approach 1:

The machine learning model automatically analyzes the input image and generates augmentation decisions without requiring user input or interaction. The system serves itself by autonomously determining which augmentations to apply based on image content analysis, thereby eliminating user burden while maintaining customization.

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The patent replaces the manual user interaction mechanism with an automated machine learning-based decision system. Instead of users manually selecting augmentations, the system uses computer vision and machine learning models to automatically generate augmentation decisions, substituting mechanical user input with intelligent automation.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

2Ease of operation

If machine learning models are used to generate image augmentations automatically, then ease of operation is improved, but device complexity increases

Engineering Contradiction:
Improveease of useVSAvoidsystem complexity
Core Design Contradiction:
Ease of operationVSDevice complexity

Solution Approach 1:

The patent introduces machine learning models as intermediary components between the input image and the augmentation application. These models act as mediators that automatically analyze image content and generate augmentation decisions, simplifying the user interface while managing system complexity through modular architecture.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The system segments the augmentation process into distinct functional components: image input, machine learning analysis, decision generation, and augmentation application. This segmentation allows each component to be optimized independently, managing overall system complexity while enabling automated operation.

Inventive Principle:
Principle #1Segmentation

3Manufacturing precision

If contextual analysis is performed using machine learning, then realism and quality of augmentations are improved, but processing time and computational resources increase

Engineering Contradiction:
Improveaugmentation qualityVSAvoidprocessing time
Core Design Contradiction:
Manufacturing precisionVSLoss of time

Solution Approach 1:

The machine learning models perform preliminary analysis of the input image to pre-determine augmentation decisions before the actual augmentation is applied. This preliminary action enables the system to prepare augmentation strategies in advance, reducing real-time processing delays while maintaining high augmentation quality.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS12165244B2Machine learning in augmented reality content items
Publication Date: 2024.12.10 SNAP INC
  • US12165244B2 patent drawing
  • US12165244B2 patent drawing
  • US12165244B2 patent drawing

AI summary

Systems and methods herein describe receiving an image via an image capture device, using a machine learning model, generating an image augmentation decision, accessing an augmented reality content item, associating the generated image augmentation decision with the augmented reality content item, modifying the received image using the augmented reality content item and the associated image augmentation decision, and causing presentation of the modified image on a graphical user interface of a computing device.