Image Object Segmentation Using Reference Patches
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image segmentation techniques are inefficient and ineffective, particularly for manual selection methods which are time-consuming and prone to errors, and automated techniques that rely on saliency or category-specific models fail in cluttered images or for generic object categories.
Innovation Solution
An automated image segmentation method that references a dataset of already-segmented images to identify and use masks for selectively chosen patches, allowing for accurate segmentation even in cluttered images and without specialized models, by leveraging the similarity of local shapes across patches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual selection is used to segment objects, then segmentation accuracy can be achieved, but it requires significant time and user effort
Solution Approach 1:
The patent copies segmentation masks from reference images that are similar to the target image. By finding images with similar content and copying their pre-computed masks, the system achieves accurate segmentation without requiring manual selection, thus resolving the contradiction between accuracy and time consumption.
Solution Approach 2:
The patent performs preliminary segmentation by pre-computing masks for reference images in advance. These pre-computed masks are then reused for target images through similarity matching, eliminating the need for repeated manual or computational segmentation efforts and significantly reducing time consumption.
2Productivity
If saliency-based automated techniques are used, then processing speed is improved, but they fail for cluttered images with low foreground/background contrast
Solution Approach 1:
Instead of relying on saliency detection that fails for cluttered images, the patent copies masks from reference images that are similar to the target image. This approach maintains processing speed while achieving reliable segmentation results even in cluttered scenarios where saliency methods fail.
Solution Approach 2:
The patent introduces reference images as an intermediary between the target image and the segmentation result. By using similar reference images with known correct masks, the system bridges the gap between fast automated processing and reliable segmentation results for difficult images.
3Measurement precision
If category-specific models are used, then segmentation accuracy is improved for specific object types, but the system lacks versatility for other object categories
Solution Approach 1:
The patent creates a universal segmentation system that works across all object categories by using content-based similarity matching with reference images. Instead of requiring separate category-specific models, a single system can handle any object type as long as similar reference images are available, thus achieving both accuracy and versatility.
Solution Approach 2:
The patent changes the approach from fixed category-specific models to a flexible similarity-based method. By changing the segmentation parameter from category-dependent to content-dependent, the system achieves accuracy for each specific case while maintaining versatility across all categories.
4Ease of operation
If automated segmentation techniques are used, then user effort is reduced, but computational cost increases
Solution Approach 1:
The patent performs partial segmentation by only processing the necessary reference images that are similar to the target image, rather than processing all possible reference images or performing complete automated segmentation from scratch. This reduces computational cost while still achieving automated segmentation with minimal user effort.
Data Source
AI summary
Systems and methods are disclosed herein for using one or more computing devices to automatically segment an object in an image by referencing a dataset of already-segmented images. The technique generally involves identifying a patch of an already-segmented image in the dataset based on the patch of the already-segmented image being similar to an area of the image including a patch of the image. The technique further involves identifying a mask of the patch of the already-segmented image, the mask representing a segmentation in the already-segmented image. The technique also involves segmenting the object in the image based on at least a portion of the mask of the patch of the already-segmented image.


