Planar Region Guided 3D Geometry Estimation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing techniques for estimating 3D geometry from a single 2D image often result in irregularities, particularly in planar regions, which affect the accuracy of depth and normal estimates, leading to inconsistent object insertion and editing issues.

Innovation Solution

The method involves determining planarity and edge strength to identify pixels within common planar regions, using these estimates to constrain and adjust depth and normal values, thereby minimizing irregularities and improving the accuracy of 3D geometry estimation by employing a planar region constraint that requires similar normals and consistent depths within common planar regions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If deep convolutional neural networks are trained to estimate depth and normal maps from a single image, then overall prediction accuracy is improved, but irregularities occur in planar regions affecting depth and normal consistency

Engineering Contradiction:

Improvedepth and normal estimation accuracyVSAvoidconsistency within planar regions

Core Design Contradiction:

Measurement precisionVSReliability

Solution Approach 1:

The patent applies local quality by treating planar regions differently from non-planar regions. It identifies planar regions through edge detection and continuity analysis, then applies specific constraints (planar region constraints) only to these regions to enforce normal consistency and depth planarity. This localized approach maintains overall prediction accuracy while specifically addressing irregularities in planar areas.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent implements feedback by using the predicted depth and normal maps to identify planar regions, then using these identified regions to refine and adjust the predictions. The system iteratively checks for planar region characteristics and applies corrections to ensure consistency, creating a feedback loop that improves reliability without sacrificing overall accuracy.

Inventive Principle:

Principle #23Feedback

2Adaptability or versatility

If pixel depth information is used to determine object occlusion and replacement in image editing, then realistic object insertion is enabled, but irregular depth estimates in planar regions cause inconsistent occlusion behavior

Engineering Contradiction:

Improveimage editing capabilityVSAvoidocclusion consistency

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The patent applies local quality by enforcing planar region constraints specifically in planar areas where occlusion consistency is critical. By identifying planar regions and applying specialized processing to ensure depth continuity and normal consistency within these regions, the system enables reliable occlusion determination for object insertion while maintaining overall editing versatility.

Inventive Principle:

Principle #3Local quality

3Adaptability or versatility

If pixel normal information is used to determine object orientation for insertion, then objects can be oriented to match surface geometry, but irregular normal estimates in planar regions cause inconsistent object orientations

Engineering Contradiction:

Improveobject orientation capabilityVSAvoidorientation consistency

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The patent applies local quality by enforcing planar region constraints that ensure normal consistency within identified planar regions. This localized constraint application guarantees that objects inserted into planar regions will have consistent orientations across the entire region, while preserving the ability to handle various orientation scenarios in non-planar areas.

Inventive Principle:

Principle #3Local quality

4Productivity

If single image 3D geometry estimation is performed without planar region constraints, then processing speed is maintained, but accuracy and consistency in planar regions deteriorate

Engineering Contradiction:

Improveprocessing efficiencyVSAvoidplanar region geometry accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent applies preliminary action by performing edge detection and planar region identification before final depth and normal map generation. By pre-identifying planar regions and preparing constraint masks, the system can efficiently apply planar region constraints during the estimation process without significant computational overhead, maintaining processing speed while improving planar region accuracy.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS10290112B2Planar region guided 3D geometry estimation from a single image

Publication Date: 2019.05.14 ADOBE INC

AI summary

Techniques for planar region-guided estimates of 3D geometry of objects depicted in a single 2D image. The techniques estimate regions of an image that are part of planar regions (i.e., flat surfaces) and use those planar region estimates to estimate the 3D geometry of the objects in the image. The planar regions and resulting 3D geometry are estimated using only a single 2D image of the objects. Training data from images of other objects is used to train a CNN with a model that is then used to make planar region estimates using a single 2D image. The planar region estimates, in one example, are based on estimates of planarity (surface plane information) and estimates of edges (depth discontinuities and edges between surface planes) that are estimated using models trained using images of other scenes.