Depth Image Segmentation for Rapid Object Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing three-dimensional imaging systems, including time-of-flight and phase-shift systems, face challenges in rapidly and reliably identifying objects within acquired data, especially in conditions with ambient light and reflectivity variations, which can lead to inaccurate depth measurements and difficulties in distinguishing overlapping objects.

Innovation Solution

The method segments depth images into groups of objects with logically connected pixels, using modified connected component analysis and depth value histograms to label pixels based on similar z-depth values, allowing for the identification of 'blobs' that correspond to different objects, and models these blobs as primitives or predefined shapes for rapid object recognition.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If traditional object identification methods are used on depth images, then the system can process the data, but the identification is slow and unreliable especially under ambient light and reflectivity variations

Engineering Contradiction:

Improveobject identification speedVSAvoidobject identification accuracy

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The depth image is segmented into distinct depth regions using depth value histograms and connected component analysis. Pixels are grouped into blobs based on similar depth values and spatial connectivity, creating simplified representations of objects that can be rapidly identified and tracked, thereby improving both speed and reliability of object detection

Inventive Principle:

Principle #1Segmentation

2Measurement precision

If depth images are processed pixel-by-pixel, then detailed information is captured, but computational requirements become excessively high

Engineering Contradiction:

Improvedepth measurement detailVSAvoidcomputational complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

Instead of processing each pixel individually, the image is segmented into depth-based blobs and regions. Connected component analysis groups pixels with similar depth values into unified objects, reducing the number of processing units from millions of pixels to a manageable number of objects, thereby significantly lowering computational complexity while preserving depth measurement precision

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The essential depth information is extracted and condensed into depth value histograms and region labels. By extracting the key characteristics (depth values, spatial connectivity) and representing them in compressed form, the system maintains measurement precision while reducing the data volume and computational burden

Inventive Principle:

Principle #2Taking out (Extraction)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This approach enables more accurate and rapid identification of objects within depth images, improving the reliability of object detection in various applications, such as motor vehicle safety systems, by correctly characterizing images and reducing computational requirements through blob tracking.

Implementation Method 1

A more accurate distance measuring system is a so-called time-of-flight (TOF) system. TOF system 100 can be implemented on a single IC 110, without moving parts and with relatively few off-chip components.

Methodology Applied

Scientific EffectTime of Flight: Time of Flight

Implementation Method 2

Each imaging pixel detector 140 measures both intensity or amplitude of the optical energy received, and the phase-shift of the optical energy as it travels from emitter 120, through distance Z to target object 20, and then distance again back to imaging sensor array 130.

Methodology Applied

Scientific EffectPhase-shift detection:

Implementation Method 3

Some of the emitted optical energy will be reflected off the surface of target object 20, and will pass through an aperture field stop and lens, collectively 135, and will fall upon two-dimensional array 130 of pixel detectors 140 where an image is formed.

Methodology Applied

Scientific EffectLight reflection: Reflection

Data Source

PatentUS8009871B2Method and system to segment depth images and to detect shapes in three-dimensionally acquired data

Publication Date: 2011.08.30 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

A method and system analyzes data acquired by image systems to more rapidly identify objects of interest in the data. In one embodiment, z-depth data are segmented such that neighboring image pixels having similar z-depths are given a common label. Blobs, or groups of pixels with a same label, may be defined to correspond to different objects. Blobs preferably are modeled as primitives to more rapidly identify objects in the acquired image. In some embodiments, a modified connected component analysis is carried out where image pixels are pre-grouped into regions of different depth values preferably using a depth value histogram. The histogram is divided into regions and image cluster centers are determined. A depth group value image containing blobs is obtained, with each pixel being assigned to one of the depth groups.