XR Headset Tracking for Robotic Surgery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer-assisted navigation systems in surgery face challenges with intermittent tracking pauses due to obstruction by personnel or objects, requiring improvements in tracking performance for accuracy, robustness, and ergonomics.
Innovation Solution
The implementation of extended reality (XR) headsets equipped with tracking cameras that combine tracking information with other XR headsets or auxiliary tracking bars, enabling pose chaining operations to enhance the robustness and range of motion tracking during surgical procedures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single near infrared (NIR) stereo camera setup is used for tracking, then the system structure is simple, but tracking robustness deteriorates due to obstructions by personnel or objects
Solution Approach 1:
The patent combines multiple tracking cameras from different sources (XR headsets and auxiliary tracking bars) into a unified tracking system. The camera tracking system merges tracking data from multiple cameras to determine poses of surgical instruments, patient anatomy, and equipment, thereby improving tracking robustness against obstructions while distributing system complexity across multiple independent tracking units.
2Measurement precision
If multiple tracking cameras are combined to improve tracking robustness, then tracking accuracy improves, but device complexity increases
Solution Approach 1:
The tracking system is segmented into multiple independent tracking units (XR headsets and auxiliary tracking bars), each with its own cameras. This segmentation allows the system to achieve high tracking accuracy through multiple viewpoints while managing complexity by keeping each unit relatively simple and independent.
Solution Approach 2:
XR headsets serve dual functions: providing augmented reality visualization for surgeons and simultaneously acting as tracking cameras for the navigation system. This multi-functionality contributes to tracking accuracy without proportionally increasing device complexity.
3Reliability
If a single camera setup is used, then the system is easy to operate, but tracking is prone to intermittent pauses due to obstructions
Solution Approach 1:
The patent merges multiple tracking perspectives from XR headsets and auxiliary tracking bars into a unified tracking system. This combination ensures continuous tracking by providing redundant viewpoints, so if one camera is obstructed, others can maintain tracking of surgical instruments and patient anatomy without interruption.
4Reliability
If XR headsets with tracking cameras are deployed, then tracking robustness improves, but system complexity increases
Solution Approach 1:
XR headsets perform multiple functions: they provide augmented reality visualization for surgeons and simultaneously serve as mobile tracking cameras. This multi-functionality improves tracking robustness through multiple mobile viewpoints without proportionally increasing system complexity, as the same hardware serves dual purposes.
Solution Approach 2:
The XR headsets are worn by surgical personnel who are already present in the operating room, so the tracking system utilizes the natural movement and positioning of surgical staff rather than requiring dedicated tracking equipment to be positioned throughout the space. This self-service approach improves tracking coverage without adding separate tracking infrastructure.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A surgical system including a XR headset, a tracking system, and an XR headset controller. The XR headset can be worn by a user during a surgical procedure and includes a see-through display screen configured to display a world-registered XR image and to allow at least a portion of a real-world scene to pass therethrough for viewing by the user. The tracking system can determine a real-world pose of the XR headset and a real-world pose of a real-world element. The real-world pose of the XR headset and the real-world pose of the real-world element being determined relative to a real-world coordinate system. The XR headset controller can generate the world-registered XR image based on the real-world pose of the XR headset and the real-world pose of the real-world element. The world-registered XR image includes a virtual element that is generated based on a characteristic of the real-world element.