Augmented Reality Makeup Mesh Tracking for In-Message Shopping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Messaging systems lack the ability to provide augmented reality-based makeup applications that allow users to try on virtual makeup products and seamlessly integrate shopping functionality within the messaging interface.
Innovation Solution
The system integrates augmented reality content items that simulate makeup application on facial regions, allowing users to select and try on virtual makeup products, which can be added to a shopping cart for purchase, leveraging mesh generation and machine learning for facial region detection and product recommendation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If augmented reality makeup functionality is integrated into the messaging system, then user engagement and convenience are enhanced, but device complexity and processing requirements increase
Solution Approach 1:
The system divides the augmented reality makeup functionality into separate modular components: facial region detection module, mesh generation module, makeup application module, and shopping cart integration module. Each module handles a specific task independently, allowing the complex functionality to be managed through discrete, manageable segments that can be developed and maintained separately.
Solution Approach 2:
The patent introduces an intermediary processing layer that bridges the messaging interface and the augmented reality processing. This intermediary handles the coordination between user interactions in the messaging system and the complex AR operations, absorbing much of the system complexity while presenting a simple interface to users.
2Measurement precision
If facial region detection and mesh generation are performed in real-time, then makeup application accuracy is improved, but processing time and computational resources increase
Solution Approach 1:
The system performs preliminary actions by pre-detecting and segmenting facial regions before the actual makeup application. Facial landmarks and mesh structures are generated in advance and stored for quick retrieval during makeup operations, reducing real-time processing requirements while maintaining high detection accuracy.
Solution Approach 2:
The mesh generation and facial region detection are designed to be dynamic and adaptive. The system adjusts the level of detail and processing intensity based on the specific operational context, using simplified models when high precision is not critical and more detailed models when accuracy is required, thereby optimizing the balance between precision and processing time.
3Adaptability or versatility
If multiple makeup products and regions are made selectable, then product versatility is improved, but interface complexity increases
Solution Approach 1:
The interface implements local quality by organizing makeup products and selection options according to specific facial regions (eyes, lips, cheeks, etc.). Each region presents only the relevant makeup products and controls, rather than displaying all possible options simultaneously. This localized organization allows high versatility to be achieved while keeping each local interface area simple and focused.
Data Source
AI summary
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing augmented reality-based makeup. The program and method provide for receiving a request to present augmented reality content in association with an image captured by a device camera, the image depicting a user's face; accessing an augmented reality content item applying makeup to the face, the augmented reality content configured to generate a mesh for tracking plural regions of the face and to present available makeup products with respect to the plural regions; presenting the augmented reality content item in association with the face depicted in the image; receiving user input selecting a region of the plural regions; determining a set of available makeup products corresponding to the selected region; and updating presentation of the augmented reality content item based on the set of available makeup products.