Depth Estimation for Axially Moving Objects Using Blur and Scale Analysis

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing depth mapping methods struggle to accurately determine the distance of axially moving objects in a scene, as they fail to disambiguate blur changes caused by object or camera motion from changes in camera parameters, leading to incorrect depth estimates and requiring additional costs or modifications for active projection or stereo imaging.

Innovation Solution

A method that captures two images of an axially displaced object, determines the variation in blur and scale change between them, and uses these metrics to calculate at least two motion values, which identify the object's depths and axial motion in the scene, thereby correcting for motion-induced blur changes.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If standard depth from defocus methods are applied to axially moving objects, then depth information can be extracted from blur variation, but the additional blur from axial motion causes incorrect depth estimates

Engineering Contradiction:
Improvedepth estimation accuracyVSAvoiddepth measurement reliability
Core Design Contradiction:
Measurement precisionVSReliability

Solution Approach 1:

The patent applies dynamics by making the depth estimation process adaptive to motion conditions. The system dynamically adjusts the depth estimation algorithm based on detected axial motion, switching between static and motion-compensated modes. This allows the system to maintain measurement precision even when objects are moving axially between captures.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent changes the parameters used in depth estimation by introducing motion compensation. Instead of relying solely on blur variation from focus changes, the system incorporates motion parameters to separate motion-induced blur from focus-induced blur. This parameter change enables reliable depth measurement of moving objects.

Inventive Principle:
Principle #35Parameter changes

2Measurement precision

If active depth mapping methods with projection optics are used, then depth map acquisition is enabled, but cost, weight, and power requirements increase significantly

Engineering Contradiction:
Improvedepth mapping capabilityVSAvoidprojection optics requirements
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent extracts the depth mapping capability from complex active projection systems and implements it using only passive imaging with a standard camera. By taking out the projection optics and replacing them with computational methods based on blur analysis, the system achieves depth mapping without the associated cost, weight, and power requirements.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The patent replaces the mechanical projection optics system with a computational approach using image processing algorithms. Instead of using physical projection devices to measure depth, the system uses software-based analysis of blur variation in captured images, substituting mechanical complexity with computational simplicity.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

3Measurement precision

If depth from focus method is used, then depth information can be obtained, but a relatively long scan through focus is required making it impractical for video frame rates

Engineering Contradiction:
Improvedepth information accuracyVSAvoiddepth mapping speed
Core Design Contradiction:
Measurement precisionVSProductivity

Solution Approach 1:

The patent uses partial action by capturing only two images at different focus positions rather than performing a complete focus scan. This partial sampling of the focus space is sufficient to extract depth information when combined with motion compensation, achieving both accuracy and speed requirements for video applications.

Inventive Principle:
Principle #16Partial or excessive action

Solution Approach 2:

The patent applies preliminary action by detecting axial motion before performing depth estimation. By identifying motion in advance, the system can pre-adjust the depth estimation process to account for motion-induced blur, enabling faster processing that meets video frame rate requirements while maintaining accuracy.

Inventive Principle:
Principle #10Preliminary action

4Measurement precision

If multiple cameras are used for stereo imaging, then depth can be determined using stereoscopic effect, but equipment cost, alignment difficulty, and object occlusion issues arise

Engineering Contradiction:
Improvedepth determination accuracyVSAvoidmultiple camera system requirements
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent applies universality by making a single camera perform the depth determination function that traditionally required multiple cameras. The single camera captures images at different focus positions and uses computational analysis to extract depth information, eliminating the need for multiple cameras while maintaining depth measurement capability.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent creates a computational copy of the stereo imaging function using a single camera. By analyzing blur variation and applying motion compensation algorithms, the system replicates the depth determination capability of stereo imaging without requiring physical duplication of camera hardware, thus reducing cost and complexity.

Inventive Principle:
Principle #26Copying

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This approach allows for accurate depth estimation of axially moving objects without the need for active projection or camera modifications, improving depth mapping accuracy and reducing costs and complexity.

Implementation Method 1

uses a small number of images shot at different focus positions and extracts depth information from variation in blur with object distance

Methodology Applied
Scientific EffectOptical focusing: Focusing

Data Source

PatentUS10026183B2Method, system and apparatus for determining distance to an object in a scene
Publication Date: 2018.07.17 CANON KK
  • US10026183B2 patent drawing
  • US10026183B2 patent drawing
  • US10026183B2 patent drawing

AI summary

A method of determining at least two motion values of an object moving axially in a scene. A first and second image of the object in the scene is captured with an image capture device. The object is axially displaced in the scene between the captured images with respect to a sensor plane of the image capture device. A variation in blur between the first and second captured images is determined. A scale change of the object between the first and second captured images is determined. Using the determined scale change and variation in blur, at least two motion values of the object in the scene are determined. The motion values identify the depths of the object in the first and second captured images and axial motion of the object in the scene.