Surgical Tool Positioning via 2D-to-3D Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Minimally invasive surgical procedures face challenges in accurately perceiving three-dimensional (3D) physiology due to reliance on two-dimensional (2D) images from a single camera, which can hinder surgeons' ability to navigate and perform procedures effectively, especially in complex anatomical environments.
Innovation Solution
A positional system that uses machine learning models to convert 2D images into 3D spatial data, allowing for real-time tracking of surgical tools and anatomical structures, enabling accurate measurement of distances, angles, and curvature, and distinguishing between tissue types, thereby providing valuable contextual information to surgeons.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If minimally invasive surgical procedures use a single camera to capture 2D images, then the invasiveness of the procedure is reduced, but the ability to perceive 3D physiology is degraded
Solution Approach 1:
The system transforms 2D image data into 3D spatial information by training a machine learning model to predict depth and spatial coordinates from 2D images. This allows the system to recover three-dimensional physiological structure information from two-dimensional camera images, resolving the information loss while maintaining minimal invasiveness
Solution Approach 2:
The system creates a virtual 3D copy of the surgical field by training an AI model to generate 3D spatial representations from 2D images. This virtual copy provides surgeons with depth perception and spatial understanding without requiring physical 3D imaging hardware inside the patient's body
2Device complexity
If minimally invasive surgical procedures rely on 2D images, then the complexity of equipment is reduced, but the precision of surgical navigation is degraded
Solution Approach 1:
The system replaces complex mechanical 3D imaging devices with a machine learning-based computational approach. Instead of using multiple cameras or depth sensors, the system uses a trained AI model to infer 3D spatial information from standard 2D images, significantly reducing equipment complexity while maintaining navigation precision
Solution Approach 2:
The system changes the parameter representation from direct 3D measurement to AI-predicted 3D coordinates. The machine learning model learns to map 2D image features to 3D spatial parameters, enabling precise surgical navigation through parameter transformation rather than direct measurement
3Device complexity
If surgeons use 2D images to visualize the operative field, then the simplicity of the imaging system is maintained, but the accuracy of spatial understanding is degraded
Solution Approach 1:
The system generates a virtual 3D copy of the surgical field by training an AI model to predict depth and spatial coordinates from 2D images. This virtual representation preserves spatial information that would otherwise be lost in 2D imaging, while keeping the imaging system itself simple and minimal
Data Source
AI summary
Methods and systems for tracking surgical tools in a three-dimensional (3D) space are described. An example method includes receiving image data captured by a single camera of a first surgical instrument, wherein the image data comprises a two-dimensional (2D) image of a second surgical instrument. Using a machine learning model, a three-dimensional (3D) position of the second surgical instrument is determined. The example method further includes determining a feature of the image data based on the 3D position of the second surgical instrument, and outputting an indication of the feature using a user interface.