Augmented Reality Head-Mounted Display with Eye Tracking for Surgical Awareness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current minimally invasive and robotic surgical systems face challenges in effective communication between surgeons and OR staff, limiting visibility and requiring non-optimal methods for indicating specific locations or interventions, which can lead to inefficiencies and safety issues.
Innovation Solution
A surgical system incorporating augmented reality head-mounted displays with eye tracking and gesture detection, allowing for real-time, location-centric communication and enhanced situational awareness through augmented images and virtual controls, enabling hands-free interaction and automated assistance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If verbal communication is used to indicate locations or interventions, then communication between surgeon and OR staff is possible, but the communication is not information rich and cannot effectively indicate specific locations
Solution Approach 1:
The patent introduces an intermediary system (augmented reality display with eye tracking) that mediates between the surgeon's intent and the communication of location information. The eye tracking technology acts as an automatic intermediary that detects where the surgeon is looking and transmits this spatial information to the OR staff, eliminating the need for verbal description of locations.
Solution Approach 2:
The patent replaces the mechanical/verbal communication system with an automated optical and computational system. Instead of using voice or hand gestures to indicate location, the system uses eye tracking (optical detection) and computer processing to automatically determine and communicate the surgeon's focus of attention, substituting manual communication methods with automated technology.
2Loss of information
If the surgeon releases hands from surgical tools to point out locations, then specific locations can be indicated, but the surgeon cannot maintain control of surgical instruments
Solution Approach 1:
The system enables the surgeon to indicate locations through natural eye movements without requiring manual intervention. The eye tracking system automatically captures the surgeon's gaze direction and converts it into location information, allowing the surgeon's eyes to serve the dual function of both observing the surgical field and communicating location information to the OR staff.
Solution Approach 2:
The patent substitutes manual pointing gestures or verbal instructions with automated eye tracking technology. The system replaces the need for physical hand movements to indicate location with optical detection of eye position, allowing continuous instrument control while automatically communicating spatial information.
3Object-affected harmful factors
If multiple small incisions are used for minimally invasive surgery, then patient discomfort is reduced and recovery time is improved, but visibility of internal organs and tissue is limited
Solution Approach 1:
The patent adds a new dimension to the surgical visualization by overlaying augmented reality information onto the surgeon's field of view. Instead of relying solely on the limited direct visual field through small incisions, the system projects additional spatial information, instrument locations, and anatomical data into the third dimension of visual space, allowing comprehensive awareness without larger incisions.
Solution Approach 2:
The system merges the limited direct visual field from small incisions with augmented reality information from multiple sources. By combining the real-time camera feed from the surgical site with overlaid graphical indicators of instrument positions, anatomical structures, and other relevant data, the system creates a composite view that provides both the benefits of minimal incisions and comprehensive visibility.
4Area of stationary object
If the surgical field is limited to what is observed by an endoscope, then instrument collisions outside the field of view are unobserved, but the endoscope provides the primary view of the surgical site
Solution Approach 1:
The system implements a feedback mechanism where multiple cameras positioned around the surgical site continuously monitor areas outside the endoscope's field of view. This visual feedback is processed and used to generate alerts or graphical indicators that provide information about conditions beyond the primary view, allowing the surgical team to be aware of instrument positions and potential collisions that are not directly visible through the endoscope.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure is directed to an augmented reality head mounted device worn by a user. The device includes an image capture device configured to capture an image of a surgical environment and a transparent lens configured to display an augmented image based on the image of the surgical environment. An eye tracking module coupled to the transparent lens configured to determine a direction of a gaze of an eye of the user, wherein the direction of the gaze of the eye determined by the eye tracking module is used to manipulate the augmented image.