3D Foreground Selection for Vision System Depth Map Processing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Accurately distinguishing a user from a complex background in video-game applications using vision systems is challenging due to the complexity of background features, which complicates the task of determining the user's posture and gestures for input control.

Innovation Solution

The method involves acquiring a time-resolved sequence of depth maps, selecting an area with motion, and fitting a geometric model of the subject by excluding background sections lacking coherent motion and located far from the subject's geometry, thereby isolating the foreground and improving the accuracy of posture and gesture detection.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If the entire depth map including complex background is processed, then complete scene information is captured, but the accuracy of user posture and gesture detection deteriorates due to background interference

Engineering Contradiction:

Improveaccuracy of user posture and gesture detectionVSAvoidcomplexity of background features

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The depth map is segmented into foreground region (containing user) and background region. Motion estimation is performed separately on these segments, allowing the system to focus computational resources on the foreground where user gestures occur, while excluding the complex background from posture detection analysis.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The background section is extracted and excluded from the depth map processing. By removing the background portion that causes interference, the system processes only the foreground containing the user, thereby improving detection accuracy without being overwhelmed by background complexity.

Inventive Principle:

Principle #2Taking out (Extraction)

2Productivity

If a large portion of the depth map is processed, then more comprehensive data is available, but the processing speed and computational efficiency deteriorate

Engineering Contradiction:

Improvespeed of user input detectionVSAvoidnumber of pixels to be processed

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The depth map is divided into foreground and background segments based on motion estimation. Only the foreground segment containing user gestures is processed for posture detection, significantly reducing the number of pixels that need to be analyzed while maintaining detection comprehensiveness.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Instead of processing the entire depth map, the system performs partial processing by selecting only the foreground region. This partial action approach processes fewer pixels than the full depth map would require, thereby improving processing speed and computational efficiency.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS9594430B2Three-dimensional foreground selection for vision system

Publication Date: 2017.03.14 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

A method for controlling a computer system includes acquiring video of a subject, and obtaining from the video a time-resolved sequence of depth maps. An area targeting motion is selected from each depth map in the sequence. Then, a section of the depth map bounded by the area and lying in front of a plane is selected. This section of the depth map is used for fitting a geometric model of the subject.