Virtual Makeup Removal via Histogram Matching and Landmark Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtual facial makeup systems struggle with inconsistency and realism due to pre-applied makeup, and face detection issues such as lag, shaking, and occlusions, which affect the accuracy of makeup removal and application in videos.
Innovation Solution
A method involving histogram matching and intrinsic decomposition to remove makeup from facial images by separating reflectance and shading channels, combined with facial landmark detection and tracking to improve accuracy and reduce video lag and shaking, and a neural network-based system for personalized makeup recommendations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing virtual makeup systems overlay virtual makeup on the user's face as it is, then the system is simple to operate, but the results are false, inconsistent or unrealistic when makeup is already applied
Solution Approach 1:
The system performs preliminary makeup removal by detecting and eliminating pre-applied makeup before overlaying the virtual makeup. This preliminary action ensures that the base skin tone is revealed, providing a consistent and realistic foundation for the virtual makeup application, thereby resolving the contradiction between reliability and complexity.
Solution Approach 2:
The system extracts and removes the unwanted element (pre-applied makeup) from the user's face image before applying the virtual makeup. By taking out the interfering makeup layer, the system achieves realistic results without requiring complex manual removal processes, thus balancing reliability and device complexity.
2Measurement precision
If facial landmark detection is performed on every video frame, then detection accuracy is improved, but video lag and shaking increase
Solution Approach 1:
The system performs preliminary facial landmark detection on the first video frame to establish baseline landmarks. These pre-detected landmarks are then tracked and refined in subsequent frames using less computationally intensive methods, reducing processing time while maintaining detection accuracy and minimizing video lag.
Solution Approach 2:
The system maintains continuous tracking of facial landmarks across video frames using the initially detected landmarks as a reference. This continuous tracking approach preserves detection accuracy while reducing the computational burden of performing full detection on every frame, thereby minimizing processing time and preventing video shaking.
3Reliability
If histogram matching is performed on the entire facial image, then makeup removal completeness is improved, but processing speed decreases
Solution Approach 1:
The system segments the facial image into relevant regions (such as skin areas with makeup) and applies histogram matching only to these segmented regions rather than the entire image. This segmentation approach ensures complete makeup removal from affected areas while significantly reducing processing time by excluding irrelevant regions, thus balancing reliability and productivity.
Data Source
AI summary
The present disclosure provides systems and methods for virtual facial makeup simulation through virtual makeup removal and virtual makeup add-ons, virtual end effects and simulated textures. In one aspect, the present disclosure provides a method for virtually removing facial makeup, the method comprising providing a facial image of a user with makeups being applied thereto, locating facial landmarks from the facial image of the user in one or more regions, decomposing some regions into first channels which are fed to histogram matching to obtain a first image without makeup in that region and transferring other regions into color channels which are fed into histogram matching under different lighting conditions to obtain a second image without makeup in that region, and combining the images to form a resultant image with makeups removed in the facial regions. The disclosure also provides systems and methods for virtually generating output effects on an input image having a face, for creating dynamic texturing to a lip region of a facial image, for a virtual eye makeup add-on that may include multiple layers, a makeup recommendation system based on a trained neural network model, a method for providing a virtual makeup tutorial, a method for fast facial detection and landmark tracking which may also reduce lag associated with fast movement and to reduce shaking from lack of movement, a method of adjusting brightness and of calibrating a color and a method for advanced landmark location and feature detection using a Gaussian mixture model.