Saliency Map Weighting for Colorization ANN Foreground Saturation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Colorization ANNs often generate desaturated colors for foreground objects due to training with imbalanced data and a focus on minimizing total error, which discourages the maintenance of extreme pixel values, leading to 'brownish' output images.
Innovation Solution
The use of saliency maps to train colorization ANNs, highlighting important foreground objects and allowing the network to differentiate between background and foreground pixels, thereby maintaining extreme pixel values and improving output image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the ANN is trained to minimize total error across all pixels, then the overall image quality improves, but the foreground objects lose extreme pixel values and become desaturated
Solution Approach 1:
The patent applies local quality by using a saliency map to differentiate between foreground and background regions, applying different loss weights to different spatial locations. The saliency map identifies foreground objects and assigns higher weights to their pixels in the loss function, ensuring that foreground color saturation is prioritized over background regions during training.
2Quantity of substance
If the training data includes all image regions equally, then comprehensive coverage is achieved, but foreground objects are overwhelmed by background pixels
Solution Approach 1:
The saliency map enables local quality by assigning different importance weights to different regions of the training data. Foreground pixels identified by the saliency map contribute more to the loss function than background pixels, allowing the network to focus learning on foreground color accuracy while still processing all image regions.
Solution Approach 2:
The saliency map serves as an intermediary that mediates between the training data and the loss function. It transforms the raw pixel data into a weighted representation that highlights foreground regions, allowing the network to learn from all pixels while prioritizing foreground object color accuracy.
3Ease of manufacture
If standard loss functions are used without spatial weighting, then training is computationally simple, but the output images exhibit brownish desaturated colors
Solution Approach 1:
The saliency map acts as an intermediary between the standard loss function and the training process. It modifies the loss computation by introducing spatial weighting without requiring a completely new loss function, maintaining computational efficiency while significantly improving color fidelity through targeted weighting of foreground regions.
Data Source
AI summary
Methods and systems for training and utilizing an artificial neural network (ANN) are provided. In an example method, a computing device could receive an input image comprising a plurality of channels and determine a saliency map for the input image. The computing device could also establish at least one of the plurality of channels as a training channel and at least some of the plurality of channels as one or more ground truth channels. Further, the computing device could train an ANN to predict one or more output channels from the one or more training channels, where the training involves computationally updating weights of the ANN based on a loss function that comprises a difference between the one or more output channels and the one or more ground truth channels, and where the difference is computationally biased based on values from the saliency map.