3D Face Mesh Feedback for Accurate AR Cosmetics Try-On

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing technologies for cosmetics fail to accurately detect and map facial features, leading to inaccurate product recommendations and unsatisfactory results across various contexts.

Innovation Solution

Systems and methods generate a three-dimensional face mesh using AI and computer vision, allowing users to refine and adapt facial features, integrate user feedback to improve precision, and provide personalized cosmetic recommendations and virtual try-ons.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If existing facial detection technologies are used, then the system can process images quickly, but the precision of facial feature mapping deteriorates

Engineering Contradiction:

Improvefacial feature mapping precisionVSAvoidsystem complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent implements feedback mechanisms where users can correct inaccurate facial map points by manually adjusting them. The system then uses this corrected data to retrain and improve its machine learning models, creating a continuous improvement loop that enhances mapping precision over time without increasing operational complexity

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system performs preliminary automated facial feature detection to generate initial face meshes, which then serve as starting points for user refinement. This preliminary action reduces the overall complexity by handling the bulk of detection automatically while reserving only critical adjustments for user intervention

Inventive Principle:

Principle #10Preliminary action

2Measurement precision

If automated facial detection is used, then the system operates efficiently, but the accuracy of facial feature identification deteriorates

Engineering Contradiction:

Improvefacial feature identification accuracyVSAvoidtime for facial analysis

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system performs automated facial detection as a preliminary step to generate initial face meshes quickly. These preliminary results are then refined through user feedback, allowing the system to maintain efficiency while improving accuracy through iterative correction rather than requiring full manual analysis

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

User feedback on inaccurate facial detections is collected and used to retrain machine learning models. This feedback loop progressively improves identification accuracy over time, reducing the need for extensive manual correction in future analyses

Inventive Principle:

Principle #23Feedback

3Adaptability or versatility

If generic facial mapping is used, then the system is simple to operate, but the personalization of cosmetic recommendations deteriorates

Engineering Contradiction:

Improvepersonalization capabilityVSAvoidmapping system complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system transitions from generic facial mapping to personalized mapping by identifying and analyzing specific local features of individual users. User feedback on facial map point accuracy enables the system to capture unique facial characteristics, which then inform customized cosmetic recommendations tailored to each user's specific features

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The system performs preliminary automated facial mapping to establish a baseline personalization level. User feedback then refines this personalization progressively, allowing the system to achieve high adaptability through iterative improvement rather than requiring complete manual customization from scratch

Inventive Principle:

Principle #10Preliminary action

4Measurement precision

If manual correction of facial map points is allowed, then the precision of face mesh improves, but the ease of operation deteriorates

Engineering Contradiction:

Improveface mesh precisionVSAvoiduser interface complexity

Core Design Contradiction:

Measurement precisionVSEase of operation

Solution Approach 1:

The system performs preliminary automated detection to generate reasonable initial face meshes that are already quite accurate. This reduces the burden on users, requiring only minor adjustments rather than complete manual reconstruction, thereby maintaining ease of operation while improving precision

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback mechanisms where user corrections are collected and used to automatically improve future detections. This reduces the frequency and complexity of manual interventions needed, as the system learns from previous corrections and requires less user input over time

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS20250391111A1User Feedback Interfaces for Facial Map Points to Improve and Enhance Augmented Reality for Cosmetics Try-On

Publication Date: 2025.12.25 ELC MANAGEMENT LLC

AI summary

Systems and methods generate and adapt a three-dimensional face mesh representing a user's unique facial points and features. Further, the systems and methods allow the user to refine the characterization of the user's facial points and features, via a variety of feedback mechanisms, (e.g., augmented reality). User feedback is utilized to validate and/or adapt technologies used to generate three-dimensional face meshes (e.g., by validating/adapting machine learning models), thereby iteratively improving the precision and usefulness of three-dimensional face mesh technology in a variety of contexts, such as in virtual cosmetic tutorials and in generation of personalized recommendations.