AR Headset Fluorescence Overlay for Surgical Precision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current fluorescence-based intra-operative surgical guidance systems require surgeons to constantly switch their gaze between the actual surgical site and an external display, leading to reduced precision, efficiency, and increased human error due to the need for subjective comparison of real-world and image data.
Innovation Solution
An augmented reality system that uses a headset to overlay the fluorescence image directly in the surgeon's line of sight, adjusting the image position based on distance sensors and eye tracking to ensure accurate alignment with the real-world view, eliminating the need for external displays and reducing the discrepancy between the image and actual patient view.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If an external display is used to show fluorescence images, then the image can be visualized, but the surgeon must constantly switch gaze between the external display and the actual surgical site, reducing precision and efficiency
Solution Approach 1:
The patent combines the fluorescence image display with the surgeon's direct view of the surgical site by projecting the image onto a see-through display positioned in the surgeon's line of sight. This merging of the image display and real-world view eliminates the need to switch gaze between separate displays and the surgical site, thereby maintaining surgical precision while improving ease of operation.
Solution Approach 2:
The patent introduces a see-through display as an intermediary element that allows the fluorescence image to be superimposed on the surgeon's view of the surgical site. This intermediary display acts as a mediator between the image source and the surgeon's eyes, enabling simultaneous visualization of both the image and the actual surgical area without requiring gaze switching.
2Reliability
If the surgeon compares the displayed image with the real-world view, then the target tissue can be identified, but significant time is lost and human error increases due to the subjective comparison process
Solution Approach 1:
The patent merges the fluorescence image directly with the surgeon's view of the surgical site, eliminating the separate comparison step. By overlaying the image on the see-through display in the surgeon's line of sight, the system combines image data and real-world view into a single unified visual field, thereby improving reliability while increasing procedural efficiency.
Solution Approach 2:
The system performs preliminary alignment and positioning of the fluorescence image relative to the surgical site before the surgeon needs to view it. The image is pre-registered and overlaid in the correct spatial relationship, so that when the surgeon looks through the see-through display, the image is already in its proper context, eliminating the need for real-time comparison and subjective interpretation.
3Measurement precision
If a mobile camera unit is used to capture fluorescence, then the image can be acquired, but the camera position differs from the surgeon's viewpoint, creating a discrepancy that requires continuous adjustment
Solution Approach 1:
The patent introduces a see-through display as an intermediary that allows the fluorescence image to be viewed from the surgeon's perspective. The display acts as a mediator that can present the image in the correct spatial relationship to the surgical site, eliminating the need for the camera to be positioned exactly where the surgeon's eye would be. This resolves the viewpoint discrepancy while maintaining image accuracy.
Solution Approach 2:
The patent transitions from a single viewpoint (camera position) to multiple viewpoints by using a see-through display that can present the image from any angle the surgeon needs. The display allows the surgeon to view the fluorescence image from their natural surgical position while maintaining accurate spatial relationships, effectively adding the dimension of flexible viewpoint selection to the system.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enhances surgical precision and efficiency by providing an accurate and dynamic overlay of fluorescence images onto the surgeon's direct line of sight, improving the correlation between the augmented and real-world views, thereby reducing human error and increasing procedural speed.
Implementation Method 1
Near-infrared light is then shone onto the target area exciting the molecular ligands, which in response emit light with a specific wavelength. A camera sensitive to this range of the light spectrum is then used to detect the light to form an image
Implementation Method 2
a distance sensor configured to determine a distance between the headset and the target throughout a medical procedure
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An augmented reality, AR, system (100) for use in a medical procedure is disclosed. The AR system (100) comprises an AR headset (2), and a processor (12). The AR headset (2) comprises a camera (6a, 6b), a near eye display (4a, 4b) and a distance sensor (10a, 10b). The processor (12) is configured to adjust the position of the image obtained by the camera (6a, 6b) on the display (4a, 4b) throughout the medical procedure based on changes in the distance measured by the distance sensor (10a, 10b).