Exoscope Navigation via Deep Learning Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems for navigating and positioning an exoscope during surgeries lack the precision required for accurately targeting specific locations, leading to frequent interruptions and increased risk of mistakes.
Innovation Solution
A system utilizing on-demand deep learning based segmentation to generate an augmented image with labeled regions, allowing surgeons to select desired regions via voice commands or tapping gestures, and automatically adjusting the exoscope's field of view to center on the selected region.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If surgeon-driven voice commands and hand controls are used to move the exoscope, then the surgeon can control the exoscope movement, but the positioning precision is insufficient and frequent interruptions occur
Solution Approach 1:
The system enables automatic exoscope positioning by integrating deep learning-based surgical site identification and automated navigation algorithms. The exoscope system autonomously determines its position relative to the surgical site and automatically adjusts positioning without surgeon intervention, thereby improving both positioning precision and workflow efficiency
Solution Approach 2:
The patent replaces manual mechanical control (voice commands and hand controls) with an automated computer vision and deep learning system. The system uses image processing to identify surgical sites and automatically calculates positioning adjustments, substituting mechanical surgeon operations with intelligent automated control to achieve higher precision without interruptions
2Adaptability or versatility
If the exoscope field of view crosses over the surgical area location, then the surgeon can capture different areas, but the surgical workflow is interrupted frequently
Solution Approach 1:
The system implements continuous feedback by processing real-time images from the exoscope, automatically identifying the surgical site location, and providing feedback signals to the robotic arm control system. This closed-loop feedback mechanism enables the exoscope to automatically adjust and maintain optimal field of view positioning without surgical interruptions
Solution Approach 2:
The system performs preliminary action by pre-identifying and tracking the surgical site location using deep learning algorithms before positioning adjustments are needed. The automated system proactively determines the required field of view adjustments and executes them preemptively, preventing workflow interruptions rather than reacting to them
Data Source
AI summary
A computing system and method are provided for image navigation using on-demand deep learning based segmentation. An example system may comprise an exoscope configured to capture image data from a field of view; an image segmentation module; an intent recognition module to capture a user's intent; and one or more robotic arms configured to move the exoscope. The system may receive, via the exoscope, image data relating to an image or a video stream of a surgical site; generate, via the image segmentation module, an augmented image comprising a plurality of labeled regions overlaying the surgical site; receive, via the intent recognition module, a voice command selecting a labeled region of the plurality of labeled regions; and cause, via the one or more robotic arms, a movement of the exoscope so that the selected labeled region is within the field of view of the exoscope after the movement.