XR-Surgical Navigation Co-Registration Using Gaze and Gesture Input
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional co-registration techniques for extended reality (XR) and surgical navigation systems require specialized hardware, increasing cost and complexity, and fail to leverage biomechanical inputs effectively, leading to cumbersome and inaccurate user interactions.
Innovation Solution
A surgical system utilizing a head-mounted device (HMD) with a sensing system to detect biomechanical inputs, such as gaze or gestures, for intuitive co-registration of XR and navigation systems without additional hardware, enabling alignment of physical and virtual registration objects through controllers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional co-registration techniques use specialized hardware (calibration devices with markers, trackers attached to XR headset), then co-registration accuracy is improved, but device complexity and cost increase
Solution Approach 1:
The patent extracts the co-registration function from specialized hardware (calibration devices and trackers) and implements it using only the native capabilities of the XR headset (camera and existing sensors). This eliminates the need for additional hardware components while maintaining co-registration functionality.
Solution Approach 2:
The patent makes the XR headset's camera serve multiple functions: it is used for both normal XR operation and for co-registration detection. The existing camera hardware is leveraged to detect markers and perform coordinate system alignment, eliminating the need for dedicated co-registration hardware.
2Measurement precision
If trackers are attached to XR headset for co-registration, then co-registration capability is improved, but headset weight and size increase causing discomfort
Solution Approach 1:
The patent removes the tracker component from the headset configuration. Instead of attaching external trackers to the headset, the system uses the headset's built-in camera to detect markers in the environment, thereby achieving co-registration without adding weight to the moving headset.
3Ease of operation
If manual pointer manipulation is required for point acquisition during co-registration, then co-registration functionality is achieved, but ease of operation deteriorates due to cumulative errors and user burden
Solution Approach 1:
The patent replaces the mechanical manual pointer manipulation process with an automated visual detection system. The XR headset's camera automatically detects markers and calculates coordinate transformations, eliminating the need for manual pointing and the associated cumulative errors and user burden.
Solution Approach 2:
The system performs co-registration automatically without requiring manual intervention for point acquisition. The camera-based detection system self-services the co-registration process by autonomously identifying markers and computing the necessary transformations, freeing the user from cumbersome manual operations.
Data Source
AI summary
Techniques for co-registration of extended reality and surgical navigation systems involve detection of a spatial relationship between a virtual registration object and a physical registration object (e.g., tool). The physical registration object is observed on, or through, the display of the head-mounted device (HMD). In one implementation, biomechanical control input (e.g., hand/finger or eye tracking) from the HMD user is sensed to command acquisition of the virtual registration object relative to the physical registration object. In another implementation, the virtual registration object is presented on the HMD display for alignment with the physical registration object. In some instances, a virtual guide object is presented on the HMD display to dynamically guide the user to align the physical and virtual registration objects.