3D Laparoscopic Surface Reconstruction Using Active Stereo

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current 3D laparoscopic surface reconstruction systems using structured light methods are time-consuming, prone to errors due to illumination changes, and require complex calibration, making them unsuitable for dynamic measurements and limited by single-camera single-projector configurations.

Innovation Solution

A novel 3D laparoscopic surface reconstruction system employing active stereo technique with two image-feedback channels and one pattern-projection channel, utilizing high-resolution fiber bundles and customized GRIN lenses, and a multi-step matching procedure with phase maps of different frequencies to reduce pattern requirements and enhance accuracy and efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If multiple-shot structured light method is used, then measurement precision and accuracy are improved, but image acquisition time increases making it unsuitable for dynamic surface measurement

Engineering Contradiction:
Improvemeasurement precisionVSAvoidimage acquisition time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent segments the matching process into two distinct stages: coarse matching using low-frequency phase maps to establish initial correspondences, and fine matching using high-frequency phase maps to refine precision. This segmentation allows the system to achieve high measurement precision without requiring multiple sequential shots, thereby reducing image acquisition time for dynamic surfaces.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent applies partial action by using only the necessary frequency components for each matching stage. Instead of applying full-spectrum structured light patterns repeatedly, the system selectively uses low-frequency patterns for coarse matching and high-frequency patterns only where needed for fine matching, reducing the total number of patterns required and thus the acquisition time.

Inventive Principle:
Principle #16Partial or excessive action

2Volume of moving object

If single-camera single-projector configuration is used, then probe size is reduced, but wrong matching occurs due to illumination changes and disturbances

Engineering Contradiction:
Improveprobe sizeVSAvoidmatching reliability
Core Design Contradiction:
Volume of moving objectVSReliability

Solution Approach 1:

The patent transitions from a single-image-dimensional approach to a stereo-dimensioned approach by implementing a dual-camera system. This dimensional change provides redundant viewing angles that enable robust correspondence matching even under varying illumination conditions, as the stereo geometry constrains the solution space and eliminates ambiguous matches that plague single-camera systems.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

Solution Approach 2:

The system employs feedback mechanisms where the coarse matching results from low-frequency patterns inform the fine matching process with high-frequency patterns. This hierarchical feedback approach allows the system to adaptively refine correspondences, improving matching reliability by using previous stage results to guide subsequent processing stages.

Inventive Principle:
Principle #23Feedback

3Volume of moving object

If structured light method with single-camera single-projector is used, then probe diameter is reduced, but complex operations are required including luminance compensation and projector calibration

Engineering Contradiction:
Improveprobe diameterVSAvoidoperation complexity
Core Design Contradiction:
Volume of moving objectVSEase of operation

Solution Approach 1:

The stereo-based system performs self-calibration through geometric constraints inherent in the dual-camera configuration. The epipolar geometry and fundamental matrix relationships provide built-in reference frames that eliminate the need for external calibration targets or complex luminance compensation procedures, allowing the system to self-adjust to illumination variations without manual intervention.

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The patent replaces the mechanical calibration procedures (physical calibration targets, manual projector alignment) with computational geometry-based calibration inherent to stereo vision. The system uses mathematical relationships between the two camera views to automatically establish correspondence, substituting complex mechanical adjustment operations with automated computational processes.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS10839542B2Systems and methods for 3D laparoscopic surface reconstruction
Publication Date: 2020.11.17 THE CHINESE UNIVERSITY OF HONG KONG
  • US10839542B2 patent drawing
  • US10839542B2 patent drawing
  • US10839542B2 patent drawing

AI summary

Various aspects of a systems and method for reconstructing a surface of a three-dimensional (3D) target are disclosed herein. The method may comprise projecting a sequence of patterns to the surface of the target; capturing a first stereo endoscopic image and a second stereo endoscopic image from the patterns reflected from the surface; performing a coarse matching for the captured first and second stereo endoscopic images to acquire a set of matching candidates; and performing a precise matching for the acquired set of matching candidates to acquire reconstruction pixels for reconstructing the surface.