Augmented Surgical Projection System with Real-Time Distance Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current surgical imaging systems require surgeons to divert their gaze from the surgical site to view additional visual information on external monitors, disrupting their focus and posing challenges with scaling and registration of augmented images relative to three-dimensional anatomical structures, orientation, and maintaining focus across non-planar body topologies.
Innovation Solution
An augmented surgical imaging and projection system that automatically adjusts the scale and orientation of projected visual data using real-time distance and topology measurements, employing techniques like laser range finders, structured light, and time-of-flight sensors to ensure precise registration and focus across complex body surfaces, allowing surgeons to maintain focus on the surgical site without external equipment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If augmented image information is projected directly onto the surgical field, then the surgeon can maintain focus on the surgical site without shifting gaze, but the scaling and registration of the projected image relative to three-dimensional anatomical structures becomes complex
Solution Approach 1:
The system continuously monitors the distance between the projector and the surgical site using depth sensors, and automatically adjusts the scaling of projected images based on this feedback. This closed-loop control ensures that images remain correctly scaled and registered even as the surgeon moves the projector closer or farther from the patient's anatomy.
Solution Approach 2:
The system dynamically changes the scaling parameter of projected images based on the measured distance to the surgical site. As the distance parameter changes, the image scale is automatically adjusted to maintain proper registration with the three-dimensional anatomical structures, resolving the contradiction between ease of operation and registration precision.
2Ease of operation
If the camera moves further away from the surgical site, then the surgeon has better access and visibility, but the scale of the projected image becomes smaller than the real object requiring continuous adjustment
Solution Approach 1:
Depth sensors continuously provide feedback on the camera-projector distance to the surgical site, enabling automatic real-time adjustment of image scaling. This ensures that regardless of whether the surgeon moves closer for detailed work or farther for better access, the projected images maintain correct scale and registration with the anatomical structures.
3Loss of information
If multiple imaging modalities (CT, MRI, fluorescence) are integrated into the projection system, then comprehensive surgical information is provided, but the system complexity and difficulty of precise registration increase
Solution Approach 1:
The projection system is designed as a multi-functional platform that can integrate and project various imaging modalities including CT, MRI, and fluorescence images. By using a single unified projection system rather than separate systems for each modality, the patent reduces overall system complexity while providing comprehensive surgical information through one integrated device.
Solution Approach 2:
The system uses fiducial markers as intermediaries to facilitate precise registration of multiple imaging modalities. These markers serve as common reference points that enable accurate alignment and integration of CT, MRI, and fluorescence images, simplifying the registration process despite the diversity of imaging sources.
4Loss of information
If the surgeon views additional visual information on external monitors, then comprehensive surgical data is provided, but the surgeon must disrupt focus on the surgical site
Solution Approach 1:
The system merges the surgical field view with augmented image information by projecting CT, MRI, and fluorescence images directly onto the surgical site. This combination allows the surgeon to see both the anatomical structures and the augmented information simultaneously in the same field of view, eliminating the need to shift gaze between the surgical site and external monitors while maintaining complete surgical information.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables intuitive, real-time visual feedback and precise projection of augmented information directly onto the surgical site, improving surgical efficiency by maintaining the surgeon's focus on the surgical field while providing accurate and sensitive imaging without the need for special equipment or visible delays in focus adjustments.
Implementation Method 1
the distance-measuring device is selected from the group consisting of laser range finder, laser scanning, time of flight, structured light
Implementation Method 2
the distance-measuring device is selected from the group consisting of laser range finder, laser scanning, time of flight, structured light
Implementation Method 3
a second projector configured to project at least two predefined structured light patterns in either a visible or infrared spectrum onto the physical object, wherein the at least one of a sensor and a camera detect a reflectance of the predefined light patterns
Implementation Method 4
the at least one of a sensor and a camera detect a reflectance of the predefined light patterns and the processor calculates at least one of the distance to the physical object and a topology of the physical object using the detected reflectance
Data Source
AI summary
Imaging systems projecting augmented information on a physical object that at a minimum include a processor, a memory device operably connected to the processor, a projector operably coupled to the processor, and a distance-measuring device operably connected to the processor. The memory device stores augmented image information, and the processor is configured to project augmented image information onto the physical object. The distance-measuring device is configured to measure the distance to the physical object. The processor uses distance measurement information from the distance measuring device to adjust scaling of the augmented image information. The processor provides the scale adjusted augmented image information to the projector. System can also be used for fluorescence imaging during open surgery, for endoscopic fluorescence imaging and for registration of surgical instruments.