Neural Network Image Matting for Complex Foreground Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image matting methods fail to accurately separate foreground and background in images with similar colors and textures, leading to inaccurate alpha value determination and poor matte quality, especially in complex scenes with fur, hair, or transparent objects.
Innovation Solution
A neural network system is trained to generate accurate mattes by utilizing structure and texture information in addition to color, using synthetic composite training images and refining neural networks to improve alpha value prediction and edge sharpness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing color-based matting methods are used, then the process is simple and fast, but accuracy deteriorates when foreground and background colors are similar
Solution Approach 1:
The patent combines multiple feature types (color, texture, shape, depth) into a composite feature representation that is fed to a neural network. This composite approach allows the system to maintain speed while improving accuracy by leveraging complementary information from different feature sources, resolving the contradiction between simplicity/speed and accuracy.
Solution Approach 2:
The patent replaces traditional mechanical/color-based matting algorithms with a neural network-based system that processes multiple feature types simultaneously. This substitution enables the system to achieve both speed and accuracy by using learned representations rather than hand-crafted color-only features.
2Power
If color-only propagation methods are used, then computation is efficient, but edge sharpness and detail accuracy deteriorate
Solution Approach 1:
The patent segments the matting process into multiple stages: feature extraction (color, texture, shape, depth), feature fusion, and neural network-based alpha prediction. This segmentation allows efficient processing of individual features while achieving high precision through their integrated analysis, resolving the contradiction between computation efficiency and edge sharpness.
Solution Approach 2:
The patent transitions from two-dimensional color analysis to multi-dimensional feature space by incorporating texture, shape, and depth information. This dimensional expansion enables the system to achieve sharp edges and detailed accuracy without sacrificing computational efficiency, as the neural network processes these dimensions in parallel.
3Ease of manufacture
If simple propagation methods are used, then the system is easy to implement, but performance on complex scenes with fur and hair deteriorates
Solution Approach 1:
The patent creates a universal matting system that handles diverse scenes including fur, hair, transparent objects, and similar-color foregrounds by processing multiple feature types through a single neural network architecture. This multi-functional approach maintains ease of implementation while significantly improving reliability across complex scenarios.
Solution Approach 2:
The patent changes the input parameters from simple color values to a composite set including color, texture, shape, and depth. This parameter transformation enables the system to maintain ease of implementation through a unified neural network while achieving high reliability on complex scenes by leveraging the additional parameter dimensions.
4Loss of time
If trimaps are used to reduce unknown regions, then processing time decreases, but accuracy in blended regions deteriorates
Solution Approach 1:
The patent performs preliminary feature extraction and analysis on all image regions before applying the neural network. This preliminary action on color, texture, shape, and depth features enables the system to achieve high precision in blended regions while maintaining reduced processing time through efficient feature processing and parallel computation.
Data Source
AI summary
Methods and systems are provided for generating mattes for input images. A neural network system can be trained where the training includes training a first neural network that generates mattes for input images where the input images are synthetic composite images. Such a neural network system can further be trained where the training includes training a second neural network that generates refined mattes from the mattes produced by the first neural network. Such a trained neural network system can be used to input an image and trimap pair for which the trained system will output a matte. Such a matte can be used to extract an object from the input image. Upon extracting the object, a user can manipulate the object, for example, to composite the object onto a new background.