Depth Data Shadow Filling for 3D Image Segmentation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional methods for background-foreground segmentation in images, particularly using three-dimensional data, often result in shadows with missing depth data and distortions, leading to inaccurate object detection and hidden objects due to issues like perspective shifts, camera blockages, and IR-absorbing textures.

Innovation Solution

A method that calculates a shadow band length using background and foreground depth values, focal length, and translation distance between cameras, and refines the foreground depth to accurately fill shadows, preventing leakage and improving depth data accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional depth mapping methods are used to segment foreground from background, then segmentation can be achieved, but shadow areas with missing depth data and distortions are created

Engineering Contradiction:

Improvesegmentation accuracyVSAvoiddepth data in shadow areas

Core Design Contradiction:

ReliabilityVSLoss of information

Solution Approach 1:

The patent converts the harmful shadow areas (which traditionally represent missing or incorrect depth data) into beneficial regions by calculating and filling them with appropriate background depth values. The shadow detection and filling process transforms these problematic areas into accurate depth representations, improving overall segmentation reliability while preserving hidden objects.

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

2Loss of information

If background depth values are used to fill shadow areas, then shadow filling is achieved, but accuracy is reduced and hidden objects remain undetected

Engineering Contradiction:

Improveshadow filling completenessVSAvoiddepth value accuracy

Core Design Contradiction:

Loss of informationVSMeasurement precision

Solution Approach 1:

The patent applies local quality by treating shadow areas differently from non-shadow areas. Instead of uniformly applying background depth values across the entire image, the system specifically identifies shadow regions through detection algorithms and applies depth filling only to those localized areas. This ensures that non-shadow regions maintain their original accurate depth information while shadow regions receive appropriate background depth values.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent uses partial action by selectively filling only the shadow areas rather than the entire image. The shadow detection and filling process targets specific regions where depth data is missing, leaving the rest of the image unchanged. This partial application of depth filling maintains overall image accuracy while addressing the specific problem of shadow areas.

Inventive Principle:

Principle #16Partial or excessive action

3Adaptability or versatility

If depth values are warped onto RGB image, then depth-to-color mapping is achieved, but perspective shifts cause incorrect shadow formation

Engineering Contradiction:

Improvedepth-to-color mapping capabilityVSAvoidshadow position accuracy

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The patent implements feedback by using the detected shadow areas to correct the depth-to-color mapping process. The system first performs initial depth mapping, then detects shadow regions caused by perspective shifts, and finally fills these shadow areas with correct background depth values. This feedback loop identifies and corrects errors introduced by perspective shifts, improving shadow position accuracy while maintaining the versatility of depth-to-color mapping.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS10097793B2Method and system of depth data filling of shadows for image processing

Publication Date: 2018.10.09 INTEL CORP

AI summary

Techniques described herein are related to a system, medium, and method of depth data filling of shadows for image processing comprising obtaining a location of at least one gap of missing depth data on pixel image data of at least one image, calculating a shadow band length corresponding to a size and position of the gap on the image, and calculated by using, at least in part, background and foreground depth values, and providing depth values of multiple pixel locations extending along the shadow band length.