Two-Stage GAN for Webtoon Color Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic coloring technologies for webtoons face challenges in maintaining color consistency across wide object areas and when edges cover objects, leading to incorrect coloring, especially when transitioning between different parts of an image.
Innovation Solution
A generative adversarial network (GAN)-based system is employed to generate a color image from an edge image, involving a first GAN that converts the edge image into an intermediate image and a second GAN that converts the intermediate image into a color image, with the intermediate image having an entropy level between the edge and color images, utilizing convolution and deconvolution layers, and operating within color spaces like YUV to ensure accurate color representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing automatic coloring technology is used, then productivity is improved, but color consistency deteriorates when objects are wide or covered by edges
Solution Approach 1:
The coloring process is divided into multiple stages using different GAN models. The first GAN handles edge detection and initial coloring, while the second GAN refines the coloring to ensure consistency. This segmentation allows each model to specialize in specific aspects, improving overall color consistency while maintaining productivity.
Solution Approach 2:
An intermediate image is introduced as a mediator between the edge image and the final colored image. The first GAN generates this intermediate image from the edge image, and the second GAN generates the final colored image from the intermediate image. This intermediary step helps preserve color consistency across wide objects and areas covered by edges.
2Device complexity
If a single GAN model is used for direct coloring, then device complexity is reduced, but coloring accuracy deteriorates in complex scenarios
Solution Approach 1:
The single complex GAN model is segmented into two simpler models: the first GAN for edge-based initial coloring and the second GAN for refinement. This segmentation reduces the complexity of individual models while improving overall coloring accuracy through specialized processing at each stage.
Solution Approach 2:
The first GAN performs preliminary coloring action by generating an intermediate image that captures basic color information from edge detection. This preliminary action prepares the data for the second GAN, which then performs the final coloring with higher accuracy, reducing the burden on any single model.
Data Source
AI summary
A generative adversarial network (GAN)-based system for generating a color image from an edge image includes: a first GAN training a model for converting the edge image into an intermediate image; and a second GAN training a model for converting the intermediate image into the color image, wherein an entropy of the intermediate image corresponds to a value between an entropy of the edge image and an entropy of the color image.