3D Shape Reconstruction with Dynamic Occlusion Masking

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for generating three-dimensional shape data are compromised by non-stationary obstacles, such as moving individuals, leading to defects in the generated shape data.

Innovation Solution

An image processing apparatus that specifies areas in a captured image where non-stationary obstacles may occlude the target object, generating three-dimensional shape data by integrating occluding area information with foreground shape masks using the visual hull method to suppress defects.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If three-dimensional shape data is generated using mask images from multiple imaging apparatuses, then three-dimensional shape data can be generated, but generation accuracy is reduced when obstacles occlude the target object

Engineering Contradiction:

Improvethree-dimensional shape data generation accuracyVSAvoidocclusion by obstacles

Core Design Contradiction:

Measurement precisionVSObject-affected harmful factors

Solution Approach 1:

The system performs preliminary detection of occluding objects in captured images before generating three-dimensional shape data. By identifying occluding objects and their areas in advance, the system can exclude these areas from the mask image generation process, preventing occlusion-induced errors in the final three-dimensional reconstruction

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system extracts and removes the areas corresponding to occluding objects from the mask images. By separating the occluding object areas from the target object areas, the system eliminates the harmful effect of occlusion on three-dimensional shape data generation accuracy

Inventive Principle:

Principle #2Taking out (Extraction)

2Reliability

If the method in Japanese Patent Laid-Open No. 2019-106145 is used to suppress defects from stationary structures, then defects from stationary obstacles are reduced, but defects from non-stationary obstacles like spectators cannot be suppressed

Engineering Contradiction:

Improvethree-dimensional shape data qualityVSAvoidhandling of non-stationary obstacles

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The system dynamically identifies occluding objects by analyzing captured images from multiple imaging apparatuses. Unlike static methods that only handle predetermined stationary structures, this dynamic approach can detect and adapt to any occluding object including non-stationary elements like spectators, moving equipment, or temporary obstacles

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The occluding object detection mechanism serves multiple functions: it identifies both stationary and non-stationary obstacles, works with any type of imaging apparatus arrangement, and can handle various occlusion scenarios. This universal approach extends the system's applicability beyond the limitations of methods designed only for stationary structures

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS12555394B2Image processing apparatus, method, and storage medium for generating data based on a captured image

Publication Date: 2026.02.17 CANON KK

AI summary

The image processing apparatus specifies a first area in a captured image, the first area being an area in which another object having a possibility of occluding a target object for which to generate three-dimensional shape data may exist, the other object being not an object for which to generate three-dimensional shape data, and generates the three-dimensional shape data of the target object based on an area of the other object and an area of the target object, the area of the other object being detected from the specified first area and the area of the target object being detected from the captured image.