3D Map-Based Surgical Zoom Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing medical observation systems face challenges in stabilizing the tracking of feature points during surgical operations, particularly when endoscopes are frequently moved and tissue appearance changes, leading to instability in magnifying and observing the affected region.
Innovation Solution
A medical observation system that generates a three-dimensional map of the surgical field and sets a region-of-interest frame, allowing for continuous magnification and stable observation of the affected area from a distant position, using a camera control unit with a three-dimensional information generation unit, region-of-interest setting and estimation units, and a zoom processing unit.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If electronic zoom tracking is performed onto a feature point, then the surgical field can be magnified for observation, but the tracking becomes unstable when the endoscope moves frequently or tissue appearance changes
Solution Approach 1:
The patent creates a three-dimensional map (virtual copy) of the surgical field that remains stable even when the actual surgical field changes. This virtual model serves as a reference framework that doesn't suffer from the same instability issues as direct feature point tracking, allowing the system to maintain accurate zoom positioning despite endoscope movement or tissue appearance changes.
Solution Approach 2:
The patent transitions from two-dimensional image-based feature point tracking to three-dimensional spatial mapping. By constructing a 3D map of the surgical field with depth information and spatial relationships, the system gains an additional dimension for tracking stability, enabling more robust localization that is less sensitive to appearance changes and camera movement.
2Measurement precision
If the endoscope is inserted into the body for close observation, then detailed views are obtained, but the lens becomes contaminated with stains and fogging requiring frequent cleaning
Solution Approach 1:
The patent introduces a camera control unit with three-dimensional mapping capability as an intermediary between the endoscope and the surgeon's view. This intermediary processes the visual information to generate stable magnified images, allowing the endoscope to remain at a safer distance from the surgical site while still providing detailed observation through electronic zoom and stable image processing.
3Adaptability or versatility
If the endoscope is moved frequently during surgery, then comprehensive observation of the surgical field is achieved, but feature point tracking performance becomes insufficient
Solution Approach 1:
The patent changes the fundamental parameters of tracking from relying on two-dimensional feature point coordinates to using three-dimensional spatial relationships and map-based positioning. This parameter transformation enables the system to maintain tracking accuracy even when the endoscope moves frequently, as the 3D map provides a stable reference framework that can accommodate camera movement.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A three-dimensional information generation unit (14) generates a three-dimensional map (D(X, Y, Z)) (three-dimensional information) regarding a surgical field, based on a surgical field image (K(x, y)) captured by an imaging device (42a). A region-of-interest setting unit (20) (setting unit) then sets at least one region-of-interest in the surgical field image (K(x, y)) captured at a predetermined timing. Based on the three-dimensional map (D(X, Y, Z)) and the position of the region-of-interest set by the region-of-interest setting unit (20), a region-of-interest estimation unit (22) (estimation unit) estimates an existence position of the region-of-interest from within the surgical field image (K(x, y)) captured at a timing different from the predetermined timing. Subsequently, a zoom processing unit (26) (magnified image generation unit) generates a magnified surgical field image (L(x, y)) in which the estimated region-of-interest is magnified by a predetermined magnification, and a display control unit (40) outputs at least the magnified surgical field image (L(x, y)).