Stereoscopic Visualization System for Endoscope Using Shape-from-Shading

Solution Overview

Problem

Conventional minimally invasive surgery using monoscopic endoscopes lacks depth information, making it challenging for surgeons to accurately position surgical instruments, while dual-camera endoscopes providing stereo images are expensive and less accepted.

Innovation Solution

A stereoscopic visualization system using a monoscopic endoscope and an image conversion device with a 2D-to-3D conversion unit applying shape-from-shading and depth image-based rendering algorithms to generate stereoscopic images from 2D images, without requiring a dual-lens endoscope.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If a dual-camera endoscope is used to provide stereo images, then depth perception is improved, but cost and device complexity increase significantly

Engineering Contradiction:

Improvedepth perceptionVSAvoidendoscope structure

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent creates a virtual copy of the monoscopic endoscope's imaging capability by generating a synthetic stereo image from the single real image. The depth map algorithm creates a virtual second viewpoint, effectively copying the stereo imaging function without requiring physical dual cameras. This resolves the contradiction by providing depth perception (improving measurement precision) while maintaining the simple monoscopic hardware structure (avoiding increased device complexity).

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent replaces the mechanical dual-camera system with a computational algorithm. Instead of using two physical cameras to capture stereo images, the system uses a monoscopic camera combined with depth map generation and image rendering algorithms to synthesize stereo images. This substitution eliminates the need for complex mechanical hardware while achieving the same depth perception goal.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Measurement precision

If a dual-camera endoscope is used to provide stereo images, then depth perception is improved, but cost increases

Engineering Contradiction:

Improvedepth perceptionVSAvoidcost

Core Design Contradiction:

Measurement precisionVSEase of manufacture

Solution Approach 1:

The patent creates a virtual copy of the monoscopic endoscope's imaging capability by generating a synthetic stereo image from the single real image. The depth map algorithm creates a virtual second viewpoint, effectively copying the stereo imaging function without requiring physical dual cameras. This resolves the contradiction by providing depth perception (improving measurement precision) while maintaining the simple monoscopic hardware structure (avoiding increased device complexity).

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent replaces the mechanical dual-camera system with a computational algorithm. Instead of using two physical cameras to capture stereo images, the system uses a monoscopic camera combined with depth map generation and image rendering algorithms to synthesize stereo images. This substitution eliminates the need for complex mechanical hardware while achieving the same depth perception goal.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Ease of manufacture

If a monoscopic endoscope is used, then cost and simplicity are maintained, but depth information is lost

Engineering Contradiction:

ImprovecostVSAvoiddepth information

Core Design Contradiction:

Ease of manufactureVSLoss of information

Solution Approach 1:

The patent transforms 2D image data into 3D depth information by generating a depth map that adds the third dimension (depth) to the flat 2D image. The algorithm analyzes shading, gradients, and other 2D features to infer 3D surface geometry, effectively adding depth information without requiring a second camera. This resolves the contradiction by maintaining the simple monoscopic hardware (low cost) while recovering depth information through computational dimensionality enhancement.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Solution Approach 2:

The patent replaces the mechanical dual-camera system with a computational algorithm. Instead of using two physical cameras to capture stereo images, the system uses a monoscopic camera combined with depth map generation and image rendering algorithms to synthesize stereo images. This substitution eliminates the need for complex mechanical hardware while achieving the same depth perception goal.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentEP3130273B1Stereoscopic visualization system and method for endoscope using shape-from-shading algorithm

Publication Date: 2019.05.15 MEDICALTEK CO LTD

Figure 1
Figure 2

AI summary

A stereoscopic visualization system using shape from shading algorithm is an image conversion device connected between a monoscopic endoscope and a 3D monitor. The system applies the algorithm which generates a depth map for a 2D image of video frames. The algorithm first calculates a direction of a light source for the 2D image. Based upon the information of light distribution and shading for the 2D image, the depth map is generated. The depth map is used to calculate another view of the original 2D image by depth image based rendering algorithm in generation of stereoscopic images. After the new view is rendered, the stereoscopic visualization system also needs to convert the display format of the stereoscopic images for different kinds of 3D displays. Based on this method, it can replace the whole monoscopic endoscope with a stereo-endoscope system and no modification is required for the monoscopic endoscope.