Augmented Reality Instrument Tracking for Minimally Invasive Surgery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems for orientation support and display of medical instruments within the body, such as endoscopes, require expensive and complex setups involving CT scans and radiation, which increase patient and staff exposure to radiation and are not suitable for real-time, intuitive orientation, especially in emergency situations.
Innovation Solution
A system utilizing a camera device for image acquisition, a position measuring system with sensor units to track the instrument's 3D position, and an evaluation unit to generate a virtual image of the instrument in augmented reality, displayed in real-time on a monitor or projected directly onto the patient's body, reducing the need for preoperative CT scans and minimizing radiation exposure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If CT scans are used to create virtual images of the body interior, then spatial orientation information is provided, but radiation exposure to patient and staff increases
Solution Approach 1:
The patent uses optical cameras to capture images of the body surface and creates virtual 3D models by processing these surface images, rather than using CT scans to directly image the body interior. This copying approach through surface imaging avoids the harmful radiation exposure while still providing the necessary spatial orientation information.
Solution Approach 2:
The patent replaces the radiation-based CT scanning system with an optical imaging system using cameras. This substitution eliminates the harmful radiation component while maintaining the ability to acquire and process spatial information through a different physical principle (optical imaging instead of X-ray tomography).
2Loss of information
If CT scans are performed before surgery, then preoperative planning is enabled, but time for preparation is consumed
Solution Approach 1:
The system enables preliminary action by capturing body surface images and generating virtual 3D models before the surgical procedure. This allows preoperative planning and spatial orientation preparation to be completed in advance without requiring time-consuming CT scans, thus reducing the preparation time bottleneck.
3Productivity
If augmented reality display is implemented, then real-time orientation is improved, but system complexity increases
Solution Approach 1:
The patent makes the system more universal by using standard optical cameras and image processing technology that can be applied to various surgical scenarios. This multi-functional approach, based on general-purpose imaging components rather than specialized complex equipment, reduces the barrier to implementation while maintaining real-time orientation capabilities.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The invention relates to a system for orientation support and display of an instrument 1, equipped with one or more sensor units 14, which is inserted into or located in natural or artificially created cavities (human, animal, object), e.g., in the body of a human 1. For this purpose, multiple measurements of the 3D position of the instrument 1 equipped with one or more sensor units 14 are carried out using a position measurement system 4, so that a precise orientation and positioning of the instrument 1 in the body 2 can be calculated. Furthermore, the 3D position data 2.4 are used to simultaneously calculate a virtual image of the instrument 1. The virtual images are then projected directly onto the body surface 2.a of a human 2 using a projection unit, or superimposed onto a body surface image (real camera image of the patient) on a monitor 6 (virtual or augmented reality).The system is particularly suitable for displaying a medical instrument 1, such as a catheter or rigid or flexible endoscope, inside the body of a person 2, in real time, extracorporeally, in the correct position for a user.