Augmented Reality Depth Sensor Registration for Medical Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality systems in medical settings lack accuracy and field of view, often failing to provide clear feedback when the line of sight to tracking arrays is obstructed or the field of view is off-target, leading to user confusion.
Innovation Solution
The integration of depth sensors with tracking devices in augmented reality devices to register the device's position relative to objects in a medical environment, allowing for the display of augmentation information that addresses field of view, tracking accuracy, and object presence, thereby enhancing user understanding and system functionality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If augmented reality systems are used in medical settings, then user interface and visualization are improved, but tracking accuracy and field of view are insufficient
Solution Approach 1:
The patent combines multiple tracking technologies (optical tracking, electromagnetic tracking, and inertial measurement units) within the augmented reality system to achieve both improved user interface and maintained tracking accuracy. The fusion of these tracking methods allows the system to leverage the strengths of each technology while compensating for their individual weaknesses.
Solution Approach 2:
The patent introduces intermediary components such as reflective markers and tracking arrays that mediate between the augmented reality device and the medical environment. These intermediaries enable accurate tracking by providing reference points for optical and electromagnetic tracking systems without interfering with the augmented reality visualization.
2Measurement precision
If additional tracking technologies are integrated, then tracking accuracy is improved, but device complexity increases
Solution Approach 1:
The patent designs the augmented reality device with multi-functional components that serve multiple purposes. For example, the camera system is used both for augmented reality visualization and for optical tracking, while inertial sensors provide both motion tracking and stabilization for the augmented reality display. This reduces overall system complexity despite integrating multiple tracking technologies.
Solution Approach 2:
The system implements self-calibration and self-registration capabilities that automatically adjust for drift and alignment issues without requiring manual intervention. The augmented reality device continuously refines its tracking accuracy using feedback from multiple sensors and the environment, reducing the need for complex external calibration equipment.
3Area of stationary object
If depth sensors are used to map surroundings, then field of view is improved, but processing requirements and device complexity increase
Solution Approach 1:
The patent implements partial depth mapping by focusing computational resources on mapping only the critical regions of the surgical field rather than the entire environment. The depth sensor selectively scans areas where precision is most needed, such as near the surgical site, while using coarser sampling in peripheral regions. This approach provides sufficient field of view and depth information without requiring excessive processing power.
Solution Approach 2:
The patent divides the field of view into multiple depth zones with different processing requirements. Critical zones near the surgical site receive intensive depth mapping and processing, while peripheral zones use simplified depth estimation. This segmentation allows the system to maintain accurate depth perception where needed while reducing overall processing demands.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution provides clear and accurate feedback to users, ensuring that augmentation information is displayed in the correct context and location, improving the usability and effectiveness of medical augmented reality systems by addressing issues of obstructed views and accuracy.
Implementation Method 1
a distance measurement unit (depth sensor) of the augmented reality device to determine a position of the augmented reality device relative to an object in a medical environment
Data Source
AI summary
The disclosed method encompasses registering an augmented reality device such as augmented reality glasses with a tracking coordinate system associated with a position tracking system. This may be effected by different approaches, for example by using a distance measurement unit (depth sensor) of the augmented reality device to determine a position of the augmented reality device relative to an object in a medical environment such as in surgery or radiotherapy/radiosurgery. The object may additionally be tracked by the position tracking system so that on the basis of the distance measurement, a relative position between the augmented reality device and the tracking coordinate system can be determined in order to register the augmented reality device with the position tracking system. This registration allows displaying augmentation information in a desired context and/or at a desired location in the image of the medical environment captured by the augmented reality device.