Deep-Learning Skin Retouching via Frequency Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic skin retouching technologies in photography are inefficient, often resulting in over-smoothing of skin details and inability to effectively remove severe blemishes, requiring manual effort and being unsuitable for casual users.
Innovation Solution
The use of separate neural networks for high and low frequency layers of an image, assisted by a skin quality map, to automatically retouch skin in photographs, employing conditional generative adversarial networks and dilated residual networks for texture synthesis and color transformation, respectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If manual skin retouching operations are performed using professional tools, then realistic retouching results are achieved, but the process becomes tedious and time-consuming
Solution Approach 1:
The patent replaces manual mechanical editing operations with an automated neural network-based system. The neural networks automatically perform frequency separation, blemish detection, and retouching operations that previously required manual intervention, thereby reducing editing time while maintaining realistic results
Solution Approach 2:
The system enables self-service retouching through automated algorithms that independently analyze skin quality, generate retouching parameters, and apply corrections without human intervention. The neural networks self-adjust parameters based on input image characteristics, eliminating the need for manual parameter tuning
2Productivity
If conventional automatic retouching methods are used, then processing speed is improved, but over-smoothing of skin details occurs and severe blemishes cannot be effectively removed
Solution Approach 1:
The patent segments the retouching process into distinct frequency layers (low frequency for skin tone and high frequency for texture details). Separate neural networks process each layer independently, allowing selective retouching that preserves important skin details while removing blemishes, avoiding the over-smoothing effect of conventional single-stage methods
Solution Approach 2:
The system applies different retouching strategies to different regions and frequency bands. The high frequency path selectively targets blemishes while preserving texture, and the low frequency path adjusts skin tone uniformly. This localized approach maintains skin realism while effectively removing severe blemishes
3Manufacturing precision
If manual retouching operations are performed, then realistic results are achieved, but the process is beyond the skill level of most casual users
Solution Approach 1:
The system performs self-service by automatically analyzing skin quality, determining appropriate retouching parameters, and applying corrections without user intervention. Casual users simply need to upload an image, and the neural networks handle all complex retouching decisions, making professional-quality retouching accessible to non-experts
Solution Approach 2:
The patent replaces complex manual editing mechanics with automated neural network operations. The system substitutes manual frequency separation, blemish detection, and parameter adjustment with algorithmic processes, eliminating the need for users to learn professional editing techniques
Data Source
AI summary
Embodiments disclosed herein involve techniques for automatically retouching photos. A neural network is trained to generate a skin quality map from an input photo. The input photo is separated into high and low frequency layers which are separately processed. A high frequency path automatically retouches the high frequency layer using a neural network that accepts the skin quality map as an input. A low frequency path automatically retouches the low frequency layer using a color transformation generated by a second neural network and the skin quality map. The retouched high and low frequency layers are combined to generate the final output. In some embodiments, a training set for any or all of the networks is enhanced by applying a modification to an original image from a pair of retouched photos in the training set to improve the resulting performance of trained networks over different input conditions.