Image Object Segmentation Using Reference Patches

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing image segmentation techniques are inefficient and ineffective, particularly for manual selection methods which are time-consuming and prone to errors, and automated techniques that rely on saliency or category-specific models fail in cluttered images or for generic object categories.

Innovation Solution

An automated image segmentation method that references a dataset of already-segmented images to identify and use masks for selectively chosen patches, allowing for accurate segmentation even in cluttered images and without specialized models, by leveraging the similarity of local shapes across patches.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual selection is used to segment objects, then segmentation accuracy can be achieved, but it requires significant time and user effort

Engineering Contradiction:
Improvesegmentation accuracyVSAvoidtime consumption
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent copies segmentation masks from reference images that are similar to the target image. By finding images with similar content and copying their pre-computed masks, the system achieves accurate segmentation without requiring manual selection, thus resolving the contradiction between accuracy and time consumption.

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The patent performs preliminary segmentation by pre-computing masks for reference images in advance. These pre-computed masks are then reused for target images through similarity matching, eliminating the need for repeated manual or computational segmentation efforts and significantly reducing time consumption.

Inventive Principle:
Principle #10Preliminary action

2Productivity

If saliency-based automated techniques are used, then processing speed is improved, but they fail for cluttered images with low foreground/background contrast

Engineering Contradiction:
Improveprocessing speedVSAvoidsegmentation effectiveness
Core Design Contradiction:
ProductivityVSReliability

Solution Approach 1:

Instead of relying on saliency detection that fails for cluttered images, the patent copies masks from reference images that are similar to the target image. This approach maintains processing speed while achieving reliable segmentation results even in cluttered scenarios where saliency methods fail.

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The patent introduces reference images as an intermediary between the target image and the segmentation result. By using similar reference images with known correct masks, the system bridges the gap between fast automated processing and reliable segmentation results for difficult images.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Measurement precision

If category-specific models are used, then segmentation accuracy is improved for specific object types, but the system lacks versatility for other object categories

Engineering Contradiction:
Improvesegmentation accuracyVSAvoidobject category coverage
Core Design Contradiction:
Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent creates a universal segmentation system that works across all object categories by using content-based similarity matching with reference images. Instead of requiring separate category-specific models, a single system can handle any object type as long as similar reference images are available, thus achieving both accuracy and versatility.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent changes the approach from fixed category-specific models to a flexible similarity-based method. By changing the segmentation parameter from category-dependent to content-dependent, the system achieves accuracy for each specific case while maintaining versatility across all categories.

Inventive Principle:
Principle #35Parameter changes

4Ease of operation

If automated segmentation techniques are used, then user effort is reduced, but computational cost increases

Engineering Contradiction:
Improveuser effortVSAvoidcomputational cost
Core Design Contradiction:
Ease of operationVSUse of energy by moving object

Solution Approach 1:

The patent performs partial segmentation by only processing the necessary reference images that are similar to the target image, rather than processing all possible reference images or performing complete automated segmentation from scratch. This reduces computational cost while still achieving automated segmentation with minimal user effort.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentUS9607391B2Image object segmentation using examples
Publication Date: 2017.03.28 ADOBE INC
  • US9607391B2 patent drawing
  • US9607391B2 patent drawing
  • US9607391B2 patent drawing

AI summary

Systems and methods are disclosed herein for using one or more computing devices to automatically segment an object in an image by referencing a dataset of already-segmented images. The technique generally involves identifying a patch of an already-segmented image in the dataset based on the patch of the already-segmented image being similar to an area of the image including a patch of the image. The technique further involves identifying a mask of the patch of the already-segmented image, the mask representing a segmentation in the already-segmented image. The technique also involves segmenting the object in the image based on at least a portion of the mask of the patch of the already-segmented image.