AR Surgical Instrument Tracking via Staff Hand Proxy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing surgical procedures lack an effective way to provide an augmented reality interactive experience that enhances the real-world surgical environment with computer-generated sensory information, including visual, auditory, haptic, somatosensory, and olfactory feedback, to improve surgical precision and efficiency.
Innovation Solution
A surgical system that includes a camera system, unique identifiers, active sensors, and a surgical hub to track surgical staff members and instruments, displaying virtual elements on an augmented reality device to enhance the surgical field with real-time sensory feedback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If surgical instruments are tracked using traditional methods, then the tracking system is simple, but the system cannot provide comprehensive augmented reality feedback for all instruments
Solution Approach 1:
The patent uses surgical staff members as intermediaries to track instruments they are holding. Instead of equipping each instrument with a tracker, the system tracks the staff member's hand position and infers instrument position from that data, allowing comprehensive tracking without complex instrument-level monitoring
Solution Approach 2:
The system creates a virtual copy or representation of the physical surgical instrument in the augmented reality environment. This virtual instrument model receives transformation data from the staff member's tracked hand position, allowing the AR system to display and interact with instrument representations without physically tracking every instrument component
2Measurement precision
If passive tracking data alone is used to identify surgical instruments, then the system is easier to implement, but the system cannot accurately determine which specific instrument is being used
Solution Approach 1:
The system uses feedback loops where the surgical hub continuously receives passive tracking data, compares it against contextual information and hand position data, and refines its identification of which instrument is being used. This iterative process improves identification accuracy by cross-validating multiple data sources
Solution Approach 2:
The surgical hub performs multiple functions: it receives passive tracking data, receives active sensor data, compares hand positions to instrument characteristics, and determines instrument identity. This multi-functional approach consolidates complexity in a central system rather than requiring specialized components for each function
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A tracking system of operating room (OR) personnel and is configured to extrapolate the movement, position, orientation, and context of one or more of the active OR participants to determine the specific operation procedure in progress, the surgical instrument being used by the OR participant, and/or anticipated movement of the OR personnel. The system may further be configured to render a virtual element that includes an anticipated position of a surgical instrument, recommendations, guidance, warning, and surgical location information, as augmented reality (AR) content. The AR content is integrated in real-time with the live-feed of the surgical site, and aids the OR personnel in perioperative surgical procedures.