Dual-Path Portrait Stylization via Segmentation and Blending
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image-to-image translation methods struggle to separately stylize individuals and backgrounds in portrait images, often resulting in pixilation or poor integration of the person and background.
Innovation Solution
The proposed method employs a dual data path approach using generative adversarial networks (GANs) for face and background stylization, incorporating segmentation, inpainting, and machine-assisted blending to create a seamless stylized image, with the AgileGAN framework enhancing the latent space for detailed processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If known translation methods stylize the person and background together, then the processing is simple, but the result causes pixilation around the person's face or body and poor integration
Solution Approach 1:
The patent divides the portrait image into two distinct segments: the person (foreground) and the background. Two separate GAN-based stylization pipelines are applied independently to each segment. The first pipeline processes the person while the second pipeline processes the background, allowing for specialized stylization of each region without causing pixilation or poor integration issues that occur when treating the entire image uniformly.
2Manufacturing precision
If separate stylization pipelines are used for person and background, then the stylization quality improves, but the blending and integration become more complex
Solution Approach 1:
The patent introduces an intermediary blending mechanism that combines the outputs of the two separate stylization pipelines. A blending module uses a composite mask (combining face mask and body mask) to selectively integrate the stylized person and stylized background. This intermediary blending step resolves the complexity by providing a systematic method to merge the separately processed regions while maintaining visual coherence and avoiding artifacts.
3Ease of manufacture
If the entire image is stylized as one unit, then the processing is straightforward, but the color consistency between person and background deteriorates
Solution Approach 1:
The patent applies local quality by allowing different stylization characteristics for different regions of the image. The person and background are stylized with different GAN models and parameters optimized for their respective characteristics. The blending module then ensures color consistency by using the composite mask to harmonize the colors at the boundaries between the stylized person and background, achieving both regional specialization and global color coherence.
Data Source
AI summary
Systems and method directed to generating a stylized image are disclosed. In particular, the method includes, in a first data path, (a) applying first stylization to an input image and (b) applying enlargement to the stylized image from (a). The method also includes, in a second data path, (c) applying segmentation to the input image to identify a face region of the input image and generate a mask image, and (d) applying second stylization to an entirety of the input image and inpainting to the identified face region of the stylized image. Machine-assisted blending is performed based on (1) the stylized image after the enlargement from the first data path, (2) the inpainted image from the second data path, and (3) the mask image, in order to obtain a final stylized image.