Deep Learning Coloring System with Memory Network for Rare Color Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning-based coloring models require large amounts of data for training, which is limited for animation and cartoon content, and tend to ignore rare colors, leading to inappropriate coloring of supporting characters.
Innovation Solution
A deep learning-based coloring system utilizing a memory network with a query generation unit, neighbor calculation unit, color feature determination unit, and threshold triplet loss calculation unit, which allows for training with limited data and accurate coloring of rare colors by generating color features and updating memory based on similarity thresholds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large amount of training data is used to train the deep learning-based coloring model, then the coloring accuracy is improved, but the availability of training data is limited due to copyright issues and data scarcity in animation and cartoon content
Solution Approach 1:
The method performs preliminary color segmentation and feature extraction from limited training data before model training. By pre-processing the scarce animation/cartoon data to extract color features, spatial information, and color transitions, the system maximizes the utilization of limited training data, enabling effective model training without requiring large datasets
Solution Approach 2:
The patent introduces an intermediate color feature extraction layer that mediates between the input image and the coloring model. This intermediate layer extracts and represents color features in a structured manner, allowing the model to learn from limited data more effectively by focusing on essential color characteristics rather than raw pixel data
2Ease of manufacture
If the deep learning-based coloring model is trained to select the most frequently appearing color, then the training process is simplified, but the coloring of supporting characters becomes inappropriate due to ignoring rare colors
Solution Approach 1:
The method applies different color selection strategies to different regions of the image based on local characteristics. By analyzing spatial information and color transitions locally, the system determines whether to use dominant colors or rare colors for specific regions, ensuring that supporting characters receive appropriate rare colors while main characters receive dominant colors
Solution Approach 2:
The patent implements a dynamic color selection mechanism that adapts during the coloring process. The model dynamically adjusts color selection based on local color distribution, spatial context, and color transition patterns, rather than statically selecting only the most frequent color throughout the entire image
Data Source
AI summary
A deep learning-based coloring system includes a memory network configured to provide a color feature in response to a specific query and a coloring network configured to perform coloring, based on the color feature generated by the memory network. The memory network includes: a query generation unit configured to generate a query; a neighbor calculation unit configured to calculate k-nearest neighbors, based on similarities between the query and key memory values; a color feature determination unit configured to generate color features for indicating color information stored in the key memory; a threshold triplet loss calculation unit configured to calculate a threshold triplet loss, based on a comparison between a threshold and a distance between the color features; and a memory update unit configured to update a memory, based on whether a distance between a top value and a value of a newly input query is within the threshold.