Instance-Aware Trimap for Precise Object Boundary Editing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image segmentation models fail to accurately segment images with multiple instances of similar objects, leading to segments that are not instance-aware and are not useful for image editing operations, as they do not consider object boundaries.
Innovation Solution
An instance-aware trimap model is developed that uses an instance-aware segmentation model to generate segmented images with modified segments defined by object boundaries, allowing for precise editing of specific objects within an image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional image segmentation models are used to generate trimap segmentations, then the segmentation process is simple and fast, but the accuracy of object boundaries is poor and segments cannot distinguish between multiple instances of similar objects
Solution Approach 1:
The patent applies segmentation by dividing the image into multiple instance-specific segments rather than a single general segment. The instance-aware segmentation model separates multiple instances of similar objects (e.g., different people's hair) into distinct segments, enabling precise boundary detection for each instance while maintaining model effectiveness
Solution Approach 2:
The patent implements local quality by making the segmentation instance-aware, where each segment is locally optimized for its specific object instance. The model generates segments with precise local boundaries for each object instance rather than applying a uniform segmentation approach, improving boundary accuracy for individual objects
2Manufacturing precision
If conventional trimap segmentation is used, then the processing is efficient, but the segments include pixels from multiple objects making editing operations inaccurate
Solution Approach 1:
The patent uses segmentation to create instance-aware trimaps where foreground pixels are separated into distinct instance-specific segments. This allows editing operations to be applied precisely to individual objects (e.g., editing only one person's hair) without affecting other similar objects, improving editing precision
Solution Approach 2:
The patent introduces an instance-aware segmentation model as an intermediary between the input image and the editing operation. This intermediary generates instance-aware segments that serve as a bridge, enabling precise editing by clearly defining which pixels belong to which object instance before the editing operation is applied
Data Source
AI summary
An image editing system accesses an input image displayed via a user interface and generates an instance-aware trimap for the input image by applying an instance-aware image segmentation model to input data including the input image and a segmented image defining a segment of the input image including a first set of pixel values. The trimap defines a modified segment using a second set of pixels different from the first set of pixels. Applying the model includes detecting boundaries of an object depicted in the input image. The second set of pixels is located within the boundaries of the object. Responsive to receiving a request via the user interface, the system generates a modified image by performing an editing operation on the input image including editing a portion of the second set of pixels of the modified segment of the trimap. The system transmits, for display, the modified image.