Interactive AR Registration for Laparoscopic Surgical Navigation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Laparoscopic surgeries face challenges due to limited viewing angles and the need to mentally fuse pre-surgery imaging with live laparoscopic views, causing mental load and reduced accuracy in surgical navigation.
Innovation Solution
An augmented reality system that utilizes interactive image registration, allowing surgeons to provide supervised information for high accuracy, and re-calibrates during surgery to align pre-surgery and intra-surgery images, overlaying hidden anatomical structures and alerting surgeons to risks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If surgeons use separate monitors for pre-surgery imaging and live laparoscopic views, then diagnostic information is available, but mental load increases and surgical navigation accuracy decreases
Solution Approach 1:
The patent merges pre-surgery diagnostic imaging (CT, MRI, PET) with live laparoscopic video feeds into a single integrated augmented reality display. The system overlays registered anatomical structures from pre-surgery images onto the live surgical field, allowing surgeons to view both diagnostic information and real-time surgical views simultaneously in one field of view, eliminating the need to switch between separate monitors and reducing mental fusion requirements
2Loss of information
If surgeons use separate monitors for pre-surgery imaging and live laparoscopic views, then diagnostic information is available, but mental load increases
Solution Approach 1:
The system combines multiple information sources (pre-surgery diagnostic images and live laparoscopic video) into a single augmented reality display that presents fused anatomical information. This integration eliminates the cognitive burden of mentally aligning separate images and provides a unified view that reduces mental load while maintaining full diagnostic information availability
3Adaptability or versatility
If laparoscopic cameras are used for surgery, then minimally invasive surgery is enabled, but viewing angle is limited
Solution Approach 1:
The patent adds a third dimension to the limited 2D laparoscopic view by overlaying registered 3D anatomical structures from pre-surgery CT, MRI, or PET images onto the flat camera feed. This dimensional enhancement provides depth information and reveals hidden anatomical features that extend beyond the camera's direct line of sight, effectively expanding the viewing capability while maintaining the minimally invasive approach
4Device complexity
If image registration is performed automatically without user input, then system complexity is reduced, but registration accuracy decreases
Solution Approach 1:
The system implements an interactive feedback loop where the computer presents multiple possible matching points between pre-surgery and intra-surgery images, and the surgeon selects the correct matches based on anatomical knowledge. This user feedback refines the image registration process iteratively, achieving high accuracy while maintaining reasonable system complexity through the collaborative human-computer interface
Data Source
AI summary
This disclosure describes an interactive augmented reality system for improving surgeon's view and context awareness during laparoscopic and video assisted surgeries. Instead of purely relying on computer vision algorithms for image registration between pre-operation (or intra-operation) images/models and later intra-operation scope images, the system can implement an interactive mechanism where surgeons may provide supervised information in initial calibration phase of the augmented reality function, thus achieving high accuracy in image registration. Besides the initialization phase before operation starts, interaction between surgeon and the system can also happens during the surgery. Specifically, patient tissue might move or deform during surgery, caused by for example cutting. The augmented reality system can re-calibrate during surgery when image registration accuracy deteriorates, by seeking additional supervised labeling from surgeons. The augmented reality system can improve surgeon's view during surgery, by utilizing surgeon's guidance sporadically to achieve high image registration accuracy.