Augmented Reality Effect Rendering via Incremental Download and Smart Capture

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Augmented reality (AR) effects often take a long time to download, leading to impatient users, especially with content-rich effects, and may not render properly on low-end devices due to hardware limitations, and there is a need for efficient photo and video capture methods that balance quality and resource usage.

Innovation Solution

Implement incremental downloading of AR effect elements, prioritizing necessary components based on device capabilities and user preferences, and offer a smart capture mode that uses either native or preview capture to optimize image/video quality and resource usage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If all content packets of an AR effect are downloaded completely before display, then the AR effect quality is ensured, but the user wait time increases significantly

Engineering Contradiction:
ImproveAR effect qualityVSAvoiduser wait time
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The AR effect content is divided into multiple packets with priority levels. Critical packets containing essential rendering data are downloaded and displayed first, while non-critical packets are downloaded subsequently. This segmentation allows the system to display the AR effect progressively rather than waiting for complete download, resolving the contradiction between quality assurance and wait time reduction.

Inventive Principle:
Principle #1Segmentation

2Adaptability or versatility

If AR effects are downloaded and rendered on low-end devices, then user accessibility is improved, but rendering performance deteriorates due to hardware limitations

Engineering Contradiction:
Improvedevice compatibilityVSAvoidrendering performance
Core Design Contradiction:
Adaptability or versatilityVSReliability

Solution Approach 1:

The system identifies and prioritizes critical content packets that are essential for basic AR effect functionality. These critical packets are optimized for minimal resource requirements, enabling low-end devices to render the core AR effect. Non-critical packets that would strain device resources are deferred or omitted, allowing the AR effect to function adequately on hardware-constrained devices while maintaining acceptable performance.

Inventive Principle:
Principle #3Local quality

3Manufacturing precision

If native capture is used for photo/video capture with AR effects, then the image quality is maximized, but the processing time and resource usage increase

Engineering Contradiction:
Improveimage qualityVSAvoidprocessing time
Core Design Contradiction:
Manufacturing precisionVSLoss of time

Solution Approach 1:

The system implements a dual capture mode: preview capture for rapid, lower-resolution captures suitable for casual use, and native capture for high-quality captures when needed. Preview capture processes only the necessary AR effect data at reduced resolution, providing fast results for scenarios where maximum quality is not critical. This partial action approach reduces processing time and resource usage while maintaining acceptable quality for many use cases.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentUS10558675B2Systems and methods for capturing images with augmented-reality effects
Publication Date: 2020.02.11 META PLATFORMS INC
  • US10558675B2 patent drawing
  • US10558675B2 patent drawing
  • US10558675B2 patent drawing

AI summary

In one embodiment, a computing system captures, using a camera, a number of frames of a live scene. The system generates preview frames for an augmented scene by applying one or more augmented reality effects to the frames of the live scene. Each preview frame is based on a low-resolution image of the live scene. The low-resolution image has a lower resolution than a maximum resolution of the camera. The system stores at least one preview frame with the augmented reality effects into a storage of the computing device. The system displays a live preview of the augmented scene using the preview frames. The system receives a request from a user to capture an image of the augmented scene while the live preview is being displayed. The system retrieves, the at least one preview frame stored in the storage and outputs the retrieved at least one preview frame.