Hybrid Depth Estimation for Accurate 3D Image Display

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current image processing systems for capturing and displaying three-dimensional images often fail to accurately correlate with real-world depth, leading to decreased effectiveness in displaying meaningful information, particularly under competitive market pressures and consumer expectations.

Innovation Solution

The system processes an image sequence by forming image pairs, calculating iteration numbers for blur differences, determining zero-crossing points, and generating depth maps to create a display image that accurately represents depth, using a combination of modules for capturing, processing, and displaying images.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If multiple lenses are used to capture stereoscopic images for three-dimensional display, then depth information is enhanced, but device complexity and cost increase

Engineering Contradiction:
Improvedepth information accuracyVSAvoidsystem complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent divides the image processing into discrete grid blocks and processes depth information separately for each block. The image is segmented into multiple regions, and depth maps are calculated independently for each segment, allowing complex processing to be broken down into manageable units that reduce overall system complexity.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent introduces an intermediary computational process using blur metric analysis and iterative convolution to derive depth information from single-lens captured images. This intermediary method acts as a mediator that transforms ordinary 2D image data into 3D depth representations without requiring additional optical hardware.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Measurement precision

If complex image processing algorithms are applied to improve depth map accuracy, then depth representation quality improves, but processing time and computational resources increase

Engineering Contradiction:
Improvedepth map accuracyVSAvoidprocessing time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent performs preliminary actions by pre-calculating blur metrics for each grid block before full depth map generation. The system prepares intermediate results including blur difference calculations and iterative convolution outputs in advance, which are then used to construct the final depth map more efficiently, reducing overall processing time.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent applies partial action by focusing computational resources on calculating depth information only for regions where blur differences indicate depth variation. Instead of processing the entire image uniformly, the system selectively applies complex algorithms only where needed, based on local blur metric analysis, thereby reducing total computation time while maintaining accuracy where it matters most.

Inventive Principle:
Principle #16Partial or excessive action

3Measurement precision

If iterative convolution with blur kernel is performed to calculate blur difference, then depth measurement precision improves, but computational complexity and processing time increase

Engineering Contradiction:
Improveblur difference accuracyVSAvoidprocessing complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent segments the image into grid blocks and performs iterative convolution independently on each block rather than on the entire image. This segmentation reduces the computational complexity of each individual convolution operation while maintaining overall precision through the aggregation of block-level results into the complete depth map.

Inventive Principle:
Principle #1Segmentation

Data Source

PatentUS9639946B2Image processing system with hybrid depth estimation and method of operation thereof
Publication Date: 2017.05.02 SONY GROUP CORP
  • US9639946B2 patent drawing
  • US9639946B2 patent drawing
  • US9639946B2 patent drawing

AI summary

A system and method of operation of an image processing system includes: a capture image sequence module for receiving an image sequence; a calculate iteration depth map module for calculating an iteration depth map and a zero-crossing array; a calculate median depth map module for calculating a median depth map and a depth difference map; a calculate variance depth map module for calculating a variance depth map; a calculate image depth map module for calculating an image depth map based on the iteration depth map, the zero-crossing array, the median depth map, the depth difference map, and the variance depth map; and a calculate display image module for calculating a display image based on the received images and the image depth map for displaying on a display device.