Polarization Imaging for Transparent Object Segmentation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing computer vision systems struggle to accurately segment transparent and optically challenging objects, such as those lacking texture, as they rely on intensity images alone, leading to incorrect classifications and failures in detecting transparent objects amidst clutter or novel environments.

Innovation Solution

The use of light polarization to provide additional channels of information through polarization imaging, combined with deep learning techniques, enables the extraction of polarization feature maps that enhance the detection and segmentation of transparent and optically challenging objects by capturing unique textures and properties not visible in intensity images.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If intensity images alone are used for segmentation, then device complexity is reduced, but measurement precision and detection accuracy deteriorate for transparent objects

Engineering Contradiction:

Improvesystem complexityVSAvoidsegmentation accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The patent transitions from 2D intensity images to 4D polarization images by adding polarization angle and degree of polarization dimensions. This dimensional expansion provides additional information channels that enable accurate segmentation of transparent objects without significantly increasing overall system complexity, as the polarization data is captured simultaneously through specialized camera sensors.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Solution Approach 2:

The patent introduces polarization information as an intermediary that mediates between the light reflected from transparent objects and the segmentation algorithm. This intermediary provides additional physical properties (polarization angle and degree) that serve as distinctive features for identifying transparent objects, bridging the gap between intensity images and accurate segmentation results.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Measurement precision

If polarization imaging is used to improve segmentation accuracy, then measurement precision improves, but device complexity increases

Engineering Contradiction:

Improvesegmentation accuracyVSAvoidsystem complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent employs a polarization camera that simultaneously captures intensity and polarization information in a single imaging process. This multi-functional approach allows the system to extract multiple features (intensity, polarization angle, degree of polarization) from the same hardware setup, improving segmentation accuracy without requiring separate imaging devices for each modality.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent changes the physical parameters being measured by the imaging system from only intensity to include polarization angle and degree of polarization. This parameter expansion enables the system to capture additional physical properties of light interaction with transparent objects, providing discriminative features for accurate segmentation while using the same basic camera hardware.

Inventive Principle:

Principle #35Parameter changes

3Productivity

If traditional semantic segmentation is applied to transparent objects, then processing speed is maintained, but detection reliability deteriorates due to lack of texture information

Engineering Contradiction:

Improveprocessing speedVSAvoiddetection reliability

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent replaces reliance on mechanical texture features (surface color information, texture mapping) with optical polarization features. This substitution allows the system to detect transparent objects based on their polarization properties rather than texture, maintaining processing speed through efficient polarization feature extraction while dramatically improving detection reliability for transparent and translucent objects.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This approach significantly improves the accuracy of instance segmentation for transparent and optically challenging objects, reducing errors in cluttered scenes and novel environments, and effectively distinguishes real objects from print-out spoofs, demonstrating enhanced robustness and reliability.

Implementation Method 1

Systems and methods for transparent object segmentation using polarization cues... by using light polarization (the rotation of light waves) to provide additional channels of information

Methodology Applied

Scientific EffectLight polarization: Polarisation

Data Source

PatentUS11842495B2Systems and methods for transparent object segmentation using polarization cues

Publication Date: 2023.12.12 INTRINSIC INNOVATION LLC

AI summary

A computer-implemented method for computing a prediction on images of a scene includes: receiving one or more polarization raw frames of a scene, the polarization raw frames being captured with a polarizing filter at a different linear polarization angle; extracting one or more first tensors in one or more polarization representation spaces from the polarization raw frames; and computing a prediction regarding one or more optically challenging objects in the scene based on the one or more first tensors in the one or more polarization representation spaces.