AR Makeup Color Selection With Facial Mesh Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing messaging systems lack the ability to provide users with an immersive augmented reality experience for trying on makeup products, limiting the user's ability to visualize and purchase makeup virtually.
Innovation Solution
A messaging system that includes an augmented reality content item for tracking facial regions, allowing users to select colors and contrasts for simulated makeup overlays, and integrates a shopping cart interface for purchasing matching products.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If manual makeup application guidance is provided through graphical overlays, then users can receive instructional support, but the system lacks immersive virtual try-on capability and cannot seamlessly integrate shopping functionality
Solution Approach 1:
The system creates a virtual copy of the user's facial features through 3D mesh generation from camera images. This digital twin allows virtual makeup application without physical products, enabling immersive try-on experiences while maintaining ease of use through automated feature tracking and rendering.
Solution Approach 2:
The patent combines multiple functions into a single integrated system: facial feature detection, 3D mesh generation, virtual makeup rendering, and e-commerce shopping cart functionality. This merging allows users to try on makeup and purchase products within the same interface, enhancing both adaptability and ease of operation.
2Adaptability or versatility
If multiple makeup products and colors are made available, then user choice and customization increase, but system complexity and processing requirements increase
Solution Approach 1:
The system applies different quality levels of processing to different parts of the makeup application. High-precision 3D mesh generation is used for facial structure, while procedural textures and shader-based rendering handle makeup materials. This localized quality approach allows diverse product selection without uniformly increasing system complexity across all components.
Solution Approach 2:
The system represents makeup products through adjustable parameters (color values, texture properties, opacity, contrast) rather than fixed asset libraries. This parameter-based approach allows infinite product variations through combinatorial parameters, reducing system complexity while maintaining high adaptability and product selection variety.
3Measurement precision
If real-time facial tracking and mesh generation are implemented, then accurate makeup application is achieved, but processing time and computational resources increase
Solution Approach 1:
The system performs preliminary actions by pre-generating the 3D facial mesh and tracking key facial landmarks before makeup application. This preprocessing creates a stable geometric foundation that enables real-time rendering without recalculation, achieving high tracking accuracy while minimizing processing time during actual makeup application.
Solution Approach 2:
The facial mesh is segmented into distinct regions (forehead, cheeks, lips, eyes) with independent transformation properties. This segmentation allows parallel processing of different facial regions, maintaining high tracking precision for each region while reducing overall computational burden through distributed processing.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing make-up based augmented reality content. The program and method provide for receiving a request to present augmented reality content in association with a captured image depicting a face of the user; accessing an augmented reality content item associated with applying makeup to the face and configured to generate a mesh for tracking plural regions of the face; receiving user input selecting a region; determining at least one of a range of color values or a range of contrast values relating to available makeup products for the selected region; and presenting an interface element in association with the face, the interface element for user selection of at least one of a color value within the range of color values or a contrast value within the range of contrast values.