Dual-Spectrum Optical Tracking for Surgical Surface Acquisition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional optical tracking systems for surgical environments face challenges with lower accuracy and reliability when using visible light due to CPU-intensive image analysis and less reliable fiducial marker detection, leading to lower update rates and inaccuracies, especially in computer-assisted surgery applications.
Innovation Solution
Employing RGB color sensors and structured light modules to capture and process images in both visible and NIR spectra, enabling accurate tracking and surface reconstruction, including the use of passive fiducial markers and stereo optical systems for enhanced tracking information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If visible light image analysis is used for optical tracking, then color information and surface details are captured, but CPU-intensive processing reduces update rate and tracking accuracy
Solution Approach 1:
The patent segments the imaging task by using separate optical paths: one for visible light color imaging and another for NIR structured light projection and capture. This division allows independent optimization of each path, with the NIR path handling high-speed tracking without CPU-intensive visible light processing, thus maintaining both color information capture and high update rates
Solution Approach 2:
The patent introduces NIR structured light as an intermediary carrier that encodes spatial and surface information in a format that can be captured efficiently by image sensors. The NIR light pattern serves as a mediator between the physical surface and the digital tracking data, enabling accurate tracking without requiring intensive processing of visible light images
2Loss of information
If visible light image analysis is used for fiducial marker detection, then color-based markers can be detected, but detection reliability decreases compared to NIR systems
Solution Approach 1:
The patent uses NIR structured light patterns as an intermediary to encode fiducial marker information in a way that is more reliable for automated detection. The structured light patterns create high-contrast, geometrically defined markers that are easier to detect reliably than natural color markers, while the NIR spectrum provides better penetration and less interference from ambient lighting conditions
Solution Approach 2:
The patent changes the spectral parameter by transitioning from visible light to NIR for fiducial marker detection. This parameter change improves detection reliability because NIR light is less affected by ambient visible light interference, provides better tissue penetration, and creates higher contrast with standard surgical materials, while color information is preserved through separate visible light imaging
3Measurement precision
If NIR-based optical tracking is used, then sub-millimeter accuracy is achieved, but color information and surface texture details are lost
Solution Approach 1:
The patent merges two previously separate systems into a unified dual-mode tracking system: NIR structured light for high-precision 3D tracking and visible light color imaging for surface detail capture. The system combines the positional accuracy data from NIR tracking with the visual information from color imaging, providing both sub-millimeter precision and rich surface detail information simultaneously
Solution Approach 2:
The patent creates a multi-functional optical tracking system that can operate in multiple modes: high-precision NIR tracking mode for critical positioning tasks, color imaging mode for visual documentation and surface analysis, and combined mode that leverages both capabilities. This universal system adapts to different surgical requirements without sacrificing the advantages of either approach
4Loss of information
If full image analysis is performed for optical tracking, then comprehensive scene information is captured, but processing time increases and update rate decreases
Solution Approach 1:
The patent extracts only the essential tracking information from the captured images by using structured light patterns with known geometric properties. Instead of analyzing the entire image content, the system extracts position and orientation data from the deformation of the projected NIR patterns, significantly reducing processing time while maintaining comprehensive spatial information
Solution Approach 2:
The patent performs preliminary encoding of the tracking information by projecting structured light patterns with predetermined geometric configurations before capture. This preliminary action embeds the reference framework directly into the scene, eliminating the need for complex post-processing image analysis and enabling rapid extraction of tracking data
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach improves surgical workflow efficiency by providing real-time, accurate tracking and deeper digitalization, allowing for augmented reality overlays and enhanced surgical navigation, particularly in computer-assisted surgery systems like NAVIO®, with improved accuracy and reduced registration time.
Implementation Method 1
a structured light module to project patterns in the near-infrared (NIR) spectrum
Implementation Method 2
A first image is captured with an image sensor of at least one optical module during the projection of the pattern
Data Source
AI summary
A computer assisted system is disclosed that includes an optical tracking system and one or more computing devices. The optical tracking system includes an RGB sensor and is configured to capture color images of an environment in the visible light spectrum and tracking images of fiducials in the environment in a near-infrared spectrum. The computer assisted system is configured to generate a color image of the environment using the color images, identify fiducial locations using the tracking images, generate depth maps from the color images, reconstruct three-dimensional surfaces of structures based on the depth maps, and output a display comprising the reconstructed three-dimensional surface and one or more surgical objects that are associated with the tracked fiducials. The computer assisted system can further include a monitor or a head-mounted display (HMD) configured to present augmented reality (AR) images during a procedure.