Endoscopic 3D Model Overlays for Musculoskeletal Anatomy Navigation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Endoscopic imaging in minimally invasive surgeries, particularly in endoscopic spine surgery, faces challenges in accurately orienting within patient anatomy due to limited field of view and difficulty in identifying anatomical structures, especially when there is only one entry path for instruments and endoscopes.
Innovation Solution
A system that utilizes a data processing unit to determine the 3D position and orientation of the endoscope relative to the anatomy, augments video images with 3D model information, and employs deep neural networks to classify and outline anatomical structures, allowing for enhanced visualization and navigation through real-time tracking and virtual image generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If endoscopic imaging is used in minimally invasive surgery, then tissue damage is reduced and recovery time is shortened, but the field of view is limited and anatomical structure identification becomes difficult
Solution Approach 1:
The patent overlays 3D anatomical model information onto the 2D endoscopic video feed, adding a third dimension of spatial understanding to the limited endoscopic view. This allows surgeons to see through the narrow endoscopic field of view while maintaining awareness of broader anatomical context and instrument positions.
Solution Approach 2:
The system introduces a computer vision system and 3D anatomical model as an intermediary between the endoscope and the surgeon. This intermediary processes the limited endoscopic images and augments them with contextual anatomical information, effectively bridging the gap between the narrow field of view and comprehensive anatomical understanding.
2Object-affected harmful factors
If a single entry path is used for instruments and endoscope, then minimally invasive benefits are achieved, but orientation in patient anatomy becomes difficult
Solution Approach 1:
The system adds dimensional context by overlaying 3D anatomical models and instrument trajectory visualizations onto the 2D endoscopic view. This helps surgeons mentally orient themselves within the constrained single-port workspace by providing spatial relationships and anatomical landmarks that would otherwise be invisible through the narrow endoscopic field of view.
Solution Approach 2:
The computer vision system continuously tracks instrument positions and provides real-time feedback by highlighting anatomical structures and predicting instrument trajectories. This closed-loop feedback helps surgeons maintain spatial awareness and orientation throughout the procedure despite the limited visual information from the single endoscopic port.
3Speed
If endoscopic video images are used alone, then real-time visualization is achieved, but anatomical structure recognition is challenging
Solution Approach 1:
The system merges real-time endoscopic video feed with pre-acquired 3D anatomical models and computer vision-based structure recognition. By combining these multiple information sources, the system maintains real-time visualization speed while significantly improving anatomical structure recognition accuracy through the叠加 of complementary data layers.
Solution Approach 2:
The system performs preliminary registration of the 3D anatomical model with the patient's actual anatomy before the surgical procedure begins. This pre-alignment ensures that when the endoscopic video is augmented with anatomical overlays during surgery, the structures are accurately identified and positioned, improving recognition precision without adding real-time processing delays.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Device and methods are provided for augmenting image information of an endoscope. The devices comprise at least a data processing unit. The endoscope is configured to provide video images of at least a part of an outer surface of an anatomy of interest in front of a distal tip thereof. The data processing unit is configured to determine a 3D position and orientation of the endoscope relative to the anatomy of interest, to receive a model of the anatomy of interest, and to augment a video image provided by the endoscope with information of the model of the anatomy of interest. The 3D position and orientation of the endoscope may be determined by a tracking device and/or based on at least one X-ray image. The model of the anatomy of interest may be based on an anatomy scan.