XR Surgical Co-Registration Using HMD Gaze and Gesture Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional co-registration techniques for extended reality (XR) and surgical navigation systems require specialized hardware, increasing cost and complexity, and fail to leverage biomechanical inputs for intuitive and accurate alignment.
Innovation Solution
A surgical system utilizing a head-mounted device (HMD) with a sensing system to detect biomechanical inputs, such as gaze or gestures, for co-registering the HMD and navigation systems without additional trackers, enabling intuitive and accurate alignment through virtual registration objects and guide objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional co-registration techniques use specialized hardware (trackers, calibration devices), then co-registration accuracy is improved, but device complexity and cost increase
Solution Approach 1:
The patent extracts and eliminates the specialized tracker hardware from the co-registration system. Instead of requiring external trackers attached to the XR headset, the system uses the XR headset's built-in sensors (accelerometer, gyroscope, magnetometer, cameras) to perform co-registration, thereby removing unnecessary components while maintaining functionality
Solution Approach 2:
The XR headset performs co-registration using its own built-in sensors and processing capabilities without requiring external specialized hardware. The headset self-calibrates by using its internal sensor fusion algorithms to determine its position and orientation relative to the surgical navigation system, making the system self-sufficient
2Measurement precision
If trackers are attached to the XR headset, then co-registration capability is improved, but user comfort deteriorates due to extra mass and size
Solution Approach 1:
The patent removes the physical tracker attachment from the XR headset by utilizing the headset's built-in sensors for co-registration. This extraction eliminates the extra mass and bulk that would be imposed on the user, directly improving comfort while preserving co-registration functionality
Solution Approach 2:
The patent replaces the mechanical tracker system with an electronic sensor-based system. Instead of using physical trackers that attach to the headset and are detected by external cameras, the system uses the XR headset's internal inertial measurement unit (IMU) and other sensors to provide co-registration data, eliminating mechanical additions to the headset
3Measurement precision
If conventional techniques require manual manipulation of tracked pointer to touch pre-determined points, then co-registration is achieved, but ease of operation deteriorates due to cumbersome user interaction
Solution Approach 1:
The patent implements visual feedback through virtual guide objects that are displayed within the XR headset's field of view. These guide objects provide real-time directional cues and visual indicators that guide the user's hand movements, making the co-registration process more intuitive and easier to perform while maintaining accuracy
Solution Approach 2:
The patent introduces virtual guide objects as intermediaries between the user and the co-registration process. These virtual objects serve as visual mediators that provide step-by-step guidance, indicating which points to touch and how to position the pointer, thereby simplifying the user interaction without requiring manual manipulation of complex tracked pointers
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Techniques for co-registration of extended reality and surgical navigation systems involve detection of a spatial relationship between a virtual registration object and a physical registration object (e.g., tool). The physical registration object is observed on, or through, the display of the head-mounted device (HMD). In one implementation, biomechanical control input (e.g., hand/finger or eye tracking) from the HMD user is sensed to command acquisition of the virtual registration object relative to the physical registration object. In another implementation, the virtual registration object is presented on the HMD display for alignment with the physical registration object. In some instances, a virtual guide object is presented on the HMD display to dynamically guide the user to align the physical and virtual registration objects.