Depth Map Optimization via Superpixel Segmentation and Hole Filling

Solution Overview

Problem

Current image processing technologies face challenges in generating accurate depth information, especially in non-textured regions, due to limited reference information and inadequate algorithm complexity, leading to poor accuracy in distinguishing background and object regions.

Innovation Solution

An optimization method for image depth information that involves obtaining a to-be-repaired depth map, performing superpixel segmentation, hole filling, and statistical analysis across image segments and superpixels to generate optimized depth values, thereby improving depth map accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If simply using neighboring blocks to adjust depth information, then calculation complexity is reduced, but depth information accuracy deteriorates in large non-textured regions

Engineering Contradiction:

Improvealgorithm complexityVSAvoiddepth information accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The patent segments the depth map into multiple blocks and further divides each block into sub-blocks for independent processing. This segmentation allows the algorithm to handle different regions with appropriate complexity levels, improving depth accuracy in non-textured regions without overwhelming computational burden across the entire image.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies different processing strategies to different regions based on their characteristics. Textured regions use standard hole-filling algorithms while non-textured regions employ specialized techniques considering neighboring block depth values, ensuring optimal accuracy for each region type without uniformly increasing overall complexity.

Inventive Principle:

Principle #3Local quality

2Device complexity

If using neighboring blocks to adjust depth information, then calculation is simplified, but depth value distinction between background and object regions deteriorates

Engineering Contradiction:

Improvealgorithm complexityVSAvoiddepth value distinction

Core Design Contradiction:

Device complexityVSLoss of information

Solution Approach 1:

The patent applies different processing strategies to different regions based on their characteristics. Textured regions use standard hole-filling algorithms while non-textured regions employ specialized techniques considering neighboring block depth values, ensuring optimal accuracy for each region type without uniformly increasing overall complexity.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent extends the reference scope from local neighboring pixels to broader neighboring blocks, adding a spatial dimension to the reference information. This allows the algorithm to capture depth value patterns across larger regions, improving the distinction between background and object regions while maintaining computational efficiency.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

3Device complexity

If background and foreground have similar appearance, then image processing is simplified, but depth map accuracy deteriorates due to depth rendering interference

Engineering Contradiction:

Improveprocessing complexityVSAvoiddepth map accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The patent applies different processing strategies to different regions based on their characteristics. Textured regions use standard hole-filling algorithms while non-textured regions employ specialized techniques considering neighboring block depth values, ensuring optimal accuracy for each region type without uniformly increasing overall complexity.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent uses depth information from neighboring blocks as an intermediary reference to disambiguate regions with similar visual appearance. By referencing depth values from surrounding blocks, the algorithm can infer whether a region belongs to foreground or background even when visual cues are insufficient, improving depth map accuracy without significantly increasing processing complexity.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS10165248B2Optimization method of image depth information and image processing apparatus

Publication Date: 2018.12.25 ALTEK SEMICON

AI summary

An optimization method of image depth information and an image processing apparatus are provided. A to-be-repaired depth map generated based on a left image and a right image is obtained. A superpixel segmenting process is performed on the left image or the right image to obtain multiple superpixels. A plurality of image segments are obtained by aggregating the superpixels according to pixel information in the superpixels. A hole filling process is performed on holes of the to-be-repaired depth map to obtain a hole-filled depth map. A statistical analysis is performed on first valid depth values of the to-be-repaired depth map and second valid depth values of the hole-filled depth map to obtain a plurality of optimized depth values by using the ranges of the image segments, the ranges of the superpixels, the to-be-repaired depth map, and the hole-filled depth map.