Real-Time Virtual Makeup Augmentation via Facial Region Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual makeup/makeover systems face challenges in real-time processing efficiency, accuracy, and realism, especially on devices with limited hardware capabilities, when trying to locate and track facial features and accurately augment virtual cosmetics in live images.
Innovation Solution
A computer-implemented method that includes receiving image data from a camera, determining the location of facial features, computing characteristics of makeup products, modifying augmentation values based on these characteristics, and outputting the augmented image in real-time, using techniques such as mask data, texture images, and shader modules to enhance realism and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional virtual makeup systems are used to apply virtual cosmetics to face images, then basic augmentation functionality is provided, but processing efficiency is insufficient and realism is compromised
Solution Approach 1:
The patent segments the face into multiple defined regions (forehead, cheeks, chin, lips, eyes, nose) and processes each region independently with region-specific augmentation parameters. This segmentation enables efficient parallel processing while maintaining realistic appearance characteristics for each facial area, resolving the contradiction between processing efficiency and realism.
Solution Approach 2:
The patent applies different augmentation strategies and parameter modifications to different facial regions based on their specific characteristics. For example, lips receive color and texture modifications while eyes receive highlight and shadow adjustments. This local quality approach ensures realistic augmentation results while optimizing processing efficiency by applying only necessary modifications to each region.
2Measurement precision
If complex processing techniques are applied to achieve accurate and realistic augmentation, then augmentation quality is improved, but hardware requirements increase beyond limited device capabilities
Solution Approach 1:
The patent performs preliminary detection and localization of facial features and defined regions before applying augmentation. By pre-identifying feature locations and region boundaries, the system avoids complex real-time calculations during augmentation, enabling accurate feature location while reducing hardware requirements for devices with limited capabilities.
Solution Approach 2:
The patent modifies augmentation parameters based on detected feature characteristics and region properties rather than applying uniform complex transformations. This parameter-based approach achieves accurate and realistic augmentation by adjusting color, texture, and intensity values according to local features, while avoiding computationally intensive operations that would exceed limited device hardware capabilities.
3Loss of time
If real-time processing is implemented on devices with limited hardware capabilities, then user experience is improved, but processing accuracy and realism are compromised
Solution Approach 1:
The patent divides the face into predefined regions and processes each region independently with optimized algorithms tailored to that region's characteristics. This segmentation enables real-time processing on limited hardware by reducing the computational complexity of each sub-task, while maintaining augmentation accuracy through region-specific parameter adjustments.
Solution Approach 2:
The patent uses parameter-based augmentation that modifies color, texture, and intensity values according to detected feature characteristics rather than applying complex geometric transformations. This approach achieves both real-time processing speed and accurate augmentation results on devices with limited hardware capabilities by avoiding computationally intensive operations.
Data Source
AI summary
A system and method of augmenting image data are described. In one embodiment, the method comprises receiving data of an image captured by a camera, the captured image including a region having a visible feature of an object, storing masking data defining a plurality of masks, each mask defining a respective masked portion of the region of the captured image, sampling pixel values at predefined locations of the captured image data, selecting at least one stored mask based on the sampled pixel values, modifying pixel values in the or each selected masked portion of the region of the captured image based on colourisation parameters, and outputting the captured image with the modified pixel values for display. In other embodiments pixel values of one or more identified regions of a face in a target image are modified based on the augmentation characteristics derived from corresponding identified regions of a face in a source image.