AR Makeup Overlay via Server-Side Face Mesh Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing messaging systems lack the ability to provide users with an immersive augmented reality experience for trying on makeup products, limiting the user's interaction and engagement in virtual shopping experiences.
Innovation Solution
A messaging system that integrates augmented reality content, allowing users to select and apply virtual makeup on their faces through a messaging client, which tracks facial regions and provides simulated makeup overlays, while offering matching products for purchase.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If messaging systems provide basic text and image exchange functionality, then system simplicity and ease of operation are maintained, but user engagement and interaction depth are limited
Solution Approach 1:
The messaging system is enhanced to perform multiple functions beyond basic text and image exchange. It now includes augmented reality makeup try-on capabilities, virtual shopping experiences, and product recommendation features, allowing a single platform to serve diverse user needs while maintaining ease of operation through integrated interfaces
2Productivity
If messaging systems integrate augmented reality features for virtual makeup try-on, then user engagement and shopping experience are enhanced, but device complexity and processing requirements increase
Solution Approach 1:
A server acts as an intermediary between the messaging client and augmented reality processing. The server receives image data from the client, performs complex AR makeup overlay processing, and returns the processed images. This distributes computational complexity from the device to the server, reducing local processing requirements while maintaining enhanced user engagement
Solution Approach 2:
Complex mechanical or computational AR processing is replaced with server-side image processing algorithms. Instead of requiring sophisticated local AR engines, the system uses centralized processing resources to generate makeup overlays, simplifying the client device requirements while delivering immersive shopping experiences
3Adaptability or versatility
If messaging systems provide comprehensive product information and shopping cart functionality, then virtual shopping capability is improved, but interface complexity and information overload increase
Solution Approach 1:
The shopping interface is segmented into distinct functional modules: makeup selection, virtual try-on, product information display, and cart management. Each module operates independently with its own controls and display area, allowing comprehensive shopping functionality while maintaining interface simplicity through organized, modular presentation
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing make-up based augmented reality content. The program and method provide for receiving a request to present augmented reality content in association with a captured image depicting a face of the user; accessing an augmented reality content item associated with applying makeup to the face and configured to generate a mesh for tracking plural regions of the face; receiving user input selecting a region; determining at least one of a range of color values or a range of contrast values relating to available makeup products for the selected region; and presenting an interface element in association with the face, the interface element for user selection of at least one of a color value within the range of color values or a contrast value within the range of contrast values.