AR Makeup Segmentation Neural Network
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge lies in generating augmented reality (AR) makeup that looks realistic on a user without training data that includes images of a person with and without makeup, as existing methods struggle to train neural networks effectively in such scenarios.
Innovation Solution
The AR makeup system uses two neural networks to segment images of people with makeup, determining which parts are makeup and which are not, and then trains the second network to add AR makeup to images of users by analyzing style differences between makeup segments and images of people without makeup, using backpropagation to minimize distortions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing neural network training methods are used to generate AR makeup, then training data including images of a person with and without makeup is required, but this requirement increases data collection complexity and time consumption
Solution Approach 1:
The patent segments the makeup application process into two independent neural networks: one that segments makeup regions from images with makeup, and another that generates AR makeup on images without makeup. This segmentation allows each network to be trained independently using different data requirements, eliminating the need for paired training data and reducing data collection time while maintaining AR makeup realism.
Solution Approach 2:
The patent introduces an intermediary makeup segmentation model that acts as a bridge between the input image and the AR makeup generation process. This intermediary model extracts makeup regions from reference images, enabling the second neural network to learn makeup application patterns without requiring direct paired training data of the same person with and without makeup, thus solving the data collection bottleneck.
2Manufacturing precision
If traditional makeup application learning methods are used, then hours of learning time is required to achieve desired makeup results, but this increases time investment and complexity
Solution Approach 1:
The patent replaces the mechanical learning process of traditional makeup application with an automated neural network system. Instead of requiring users to spend hours learning manual makeup techniques, the system uses trained neural networks to automatically generate realistic AR makeup effects by analyzing reference images and applying learned patterns, thereby substituting manual skill acquisition with automated image processing while maintaining high makeup application quality.
Solution Approach 2:
The patent uses copying by having the neural network learn from existing makeup images and replicate the makeup patterns onto target images. The system captures the essence of professional makeup looks from reference images and copies them onto user images through the trained neural network, eliminating the need for users to manually learn and reproduce complex makeup techniques while achieving professional-quality results.
3Adaptability or versatility
If many makeup look images are made available on the internet, then users have more options to visualize, but the vast number of images overwhelms users and makes practical application difficult
Solution Approach 1:
The patent implements feedback by allowing users to interact with the AR makeup system in real-time. Users can try on different makeup looks generated by the neural network, see the results overlaid on their own images, and provide feedback through selections or adjustments. This feedback loop enables the system to refine and personalize makeup recommendations based on user preferences, making the vast variety of makeup looks manageable and easy to navigate while maintaining high adaptability.
Data Source
AI summary
Systems, methods, and computer readable media for messaging system with augmented reality (AR) makeup are presented. Methods include processing a first image to extract a makeup portion of the first image, the makeup portion representing the makeup from the first image and training a neural network to process images of people to add AR makeup representing the makeup from the first image. The methods may further include receiving, via a messaging application implemented by one or more processors of a user device, input that indicates a selection to add the AR makeup to a second image of a second person. The methods may further include processing the second image with the neural network to add the AR makeup to the second image and causing the second image with the AR makeup to be displayed on a display device of the user device.