Structured Light 3D Reconstruction Using Hybrid Spatio-Temporal Patterns

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing structured light systems face challenges in accurately reconstructing 3D surfaces with sharp changes in elevation due to the violation of the local smoothness approximation, leading to data loss and corruption, especially when using spatial neighborhood-based methods.

Innovation Solution

The use of a hybrid spatio-temporal pattern sequence, specifically projecting a binary de Bruijn sequence and a binary edge detection pattern, where the binary edge detection pattern's transition edges are aligned with non-transition edges of the structured light pattern, allows for accurate identification and decoding of the structured light pattern, enabling robust stereo matching and 3D surface reconstruction.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If spatial neighborhood-based methods are used for structured light reconstruction, then acquisition time is reduced making it suitable for dynamic scenes, but data loss and corruption occur in regions with sharp changes in elevation due to violation of the local smoothness approximation

Engineering Contradiction:

Improveacquisition timeVSAvoidreconstruction accuracy in sharp elevation regions

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent divides the structured light pattern into multiple code words arranged in a sequence, where each code word represents a specific spatial location. By segmenting the pattern recognition into individual code word decoding rather than relying on local smoothness assumptions, the system can accurately reconstruct points even in regions with sharp elevation changes where the local smoothness approximation would fail.

Inventive Principle:

Principle #1Segmentation

2Measurement precision

If temporal-multiplexing based methods use multiple patterns projected onto the scene, then encoding robustness increases for high accuracy reconstruction, but acquisition time increases making it more suitable for static scenes

Engineering Contradiction:

Improvereconstruction accuracyVSAvoidacquisition time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent combines the advantages of both spatial and temporal methods by projecting a single structured light pattern containing multiple unique code words that can be decoded independently. This merging approach achieves the encoding robustness of temporal methods (through unique code word identification) while maintaining the fast acquisition time of spatial methods (using only one projected pattern), thereby resolving the trade-off between accuracy and speed.

Inventive Principle:

Principle #5Merging (Combining)

3Productivity

If spatial neighborhood-based methods are used, then one unique pattern can encode the scene quickly, but the local smoothness approximation constraint requires the entire local neighborhood to be visible, leading to data loss in regions with large slopes

Engineering Contradiction:

Improveimage acquisition speedVSAvoiddata loss in sharp elevation regions

Core Design Contradiction:

ProductivityVSLoss of information

Solution Approach 1:

The patent introduces unique code words as intermediary elements that mediate between the projected structured light pattern and the reconstruction algorithm. Each code word acts as a self-contained identifier that can be decoded independently without requiring visibility of surrounding neighborhood points. This intermediary mechanism allows accurate reconstruction even when local smoothness assumptions are violated in regions with sharp elevation changes.

Inventive Principle:

Principle #24Intermediary (Mediator)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This approach enhances the accuracy of 3D surface reconstruction by effectively identifying non-transition edges within common-state sequential elements, reducing data loss and corruption, and improving the robustness of the reconstruction process, especially in regions with sharp elevations.

Implementation Method 1

projecting a structured light pattern, such as a binary de Bruijn sequence, onto a 3D surface

Methodology Applied

Scientific EffectLight: Light

Implementation Method 2

acquiring an image set of at least a portion of this projected sequence with a camera system

Methodology Applied

Scientific EffectPhotoelectric Effect: Photoelectric Effect

Data Source

PatentUS10973581B2Systems and methods for obtaining a structured light reconstruction of a 3D surface

Publication Date: 2021.04.13 7D SURGICAL ULC

AI summary

Disclosed are systems and methods for obtaining a structured light reconstruction using a hybrid spatio-temporal pattern sequence projected on a surface. The method includes projecting a structured light pattern, such as a binary de Bruijn sequence, onto a 3D surface and acquiring an image set of at least a portion of this projected sequence with a camera system, and projecting a binary edge detection pattern onto the portion of the surface and acquiring an image set of the same portion of the projected binary pattern. The acquired image set of the binary pattern is processed to determine edge locations therein, and then employed to identify the locations of pattern edges within the acquired image set of the structured light pattern. The detected edges of the structured light pattern images are employed to decode the structured light pattern and calculate a disparity map, which is used to reconstruct the 3D surface.