Neural Network Image Coloring with Confidence-Based Querying

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current methods for producing webtoon content are inefficient, particularly in coloring, which hinders the ability to keep up with the rapid expansion of the online content market and increasing user demand, leading to delays and inaccuracies in content delivery.

Innovation Solution

A method utilizing an artificial neural network to generate intermediate images, color queries, and user inputs to progressively complete the coloring of images, allowing for iterative refinement and automatic coloring, while also querying users for unclear regions to ensure accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual coloring is performed by producers, then coloring accuracy is maintained, but production time increases significantly

Engineering Contradiction:
Improvecoloring accuracyVSAvoidproduction time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The system enables self-service automatic coloring through neural networks that automatically color images without requiring manual producer intervention for routine coloring tasks, while still allowing producer oversight for quality control

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

A query system acts as an intermediary between automatic coloring and manual coloring by identifying uncertain regions and selectively querying producers only for those specific areas, rather than requiring complete manual coloring or accepting potentially inaccurate automatic coloring

Inventive Principle:
Principle #24Intermediary (Mediator)

2Productivity

If automatic coloring is performed using neural networks, then production speed increases, but coloring accuracy decreases in uncertain regions

Engineering Contradiction:
Improveproduction speedVSAvoidcoloring accuracy
Core Design Contradiction:
ProductivityVSMeasurement precision

Solution Approach 1:

The system applies automatic coloring partially - only to regions where the neural network is confident about the coloring decision. Regions with low confidence are identified and set aside for manual review, avoiding excessive automatic coloring that would reduce accuracy

Inventive Principle:
Principle #16Partial or excessive action

3Measurement precision

If producers prepare all scenes individually, then coloring quality is maintained, but content supply speed cannot keep up with market expansion

Engineering Contradiction:
Improvecoloring qualityVSAvoidcontent supply speed
Core Design Contradiction:
Measurement precisionVSSpeed

Solution Approach 1:

The coloring process is segmented into automatic coloring for confident regions and manual coloring for uncertain regions identified by the query system, allowing parallel processing and optimizing both speed and quality

Inventive Principle:
Principle #1Segmentation

Data Source

PatentUS11887224B2Method, apparatus, and computer program for completing painting of image, and method, apparatus, and computer program for training artificial neural network
Publication Date: 2024.01.30 NAVER WEBTOON LTD
  • US11887224B2 patent drawing
  • US11887224B2 patent drawing
  • US11887224B2 patent drawing

AI summary

A method of completing coloring of an image based on a query regarding a color-unknown region in the image and an answer to the query, includes: generating, by using an artificial neural network, a first intermediate image in which at least one uncolored region in a primary image is colored; generating, by using the artificial neural network, a first color query regarding the at least one color-unknown region in the primary image; and generating a secondary image based on at least one of the first color query, a first answer to the first color query, and the first intermediate image.