Template-Based AR Facial Augmentation With Real-Time Controls
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality systems are limited in their ability to provide users with real-time adjustments to pre-generated augmented reality content, restricting the range of available options for users.
Innovation Solution
An online system applies template-based facial augmentations using a facial augmentation template that includes a structure for application to a user's face and allows dynamic changes in display parameters, with a client device capturing a video feed and updating the rendering in real-time based on user input.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If pre-generated augmented reality content is used, then the content structure and display characteristics are predetermined and stable, but the ability for users to adjust the augmented reality content in real time is limited
Solution Approach 1:
The system separates the augmented reality content into two independent parts: a pre-generated template structure and real-time adjustable display parameters. The template contains the fundamental structure and geometry, while display parameters such as color, opacity, and positioning can be dynamically adjusted without regenerating the entire content, thus enabling real-time adaptation without increasing overall system complexity
Solution Approach 2:
The system transitions from static pre-generated content to dynamic content by allowing real-time modification of display parameters. The template-based approach enables the content to adapt dynamically to user preferences and contextual changes while maintaining the stability of the underlying structure, resolving the contradiction between adaptability and complexity
2Adaptability or versatility
If many different pre-generated alternatives are created to provide users with wide range of choices, then user option diversity is improved, but the system complexity and resource requirements increase significantly
Solution Approach 1:
A single template structure serves multiple functions by accepting various display parameter configurations. Instead of creating separate pre-generated content for each possible variation, the universal template can generate diverse content outcomes through parameter adjustment, significantly reducing the quantity of content alternatives needed while maintaining high diversity
Solution Approach 2:
The system achieves content diversity by changing display parameters such as color, opacity, positioning, and styling rather than creating entirely new content alternatives. This parameter-based approach allows infinite variations from a single template, eliminating the need to store and manage numerous pre-generated alternatives
3Ease of operation
If template-based facial augmentation is applied, then real-time dynamic adjustment of display parameters is enabled, but the system requires sophisticated 3D face modeling and feature point identification
Solution Approach 1:
The system introduces 3D face models and feature point identification as intermediary components that bridge the user's 2D facial appearance and the template-based augmentation. These intermediaries automatically capture facial geometry and characteristics, allowing the system to apply templates accurately without requiring users to manually configure complex parameters, thus simplifying operation while managing processing complexity
Data Source
AI summary
The present disclosure describes an online system that applies template-based facial augmentations to images as part of an augmented reality experience. A facial augmentation template is a template for an augmented reality (AR) modification to apply to a user's face. These templates include a structure for how the augmentation should be applied to the face and display parameters that a user can set for how the templates should be rendered (e.g., color, specularity, or opacity). The online system identifies feature points on a 3D model of the user's face that correspond to features on the user's actual face, and localizes the template relative to the 3D model by mapping anchor points of the template to corresponding feature points on the 3D model. The online system renders the facial augmentation template at the localization to generate an augmented image of the user's face.