Projection Mapping for Markerless Surgical Navigation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current surgical navigation systems rely on 2D displays and fiducial markers, limiting usability and requiring attention diversion from the patient, and they lack a dynamic projection mapping system for markerless subject registration and instrument tracking.
Innovation Solution
A dynamic projection mapping system that includes a sense system with optical sensors, a compute system for data processing and output generation, and a display system for projecting medical information directly onto a subject's surface, enabling markerless subject registration, instrument tracking, and real-time user collaboration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If 2D display monitors are used for surgical navigation, then the system structure is simple, but the physician must shift attention away from the patient
Solution Approach 1:
The patent introduces a projection mapping system that acts as an intermediary between the surgical navigation data and the physician's field of view. Instead of requiring the physician to look at a separate 2D monitor, the system projects relevant surgical information directly onto the patient's body or surgical field, allowing continuous visual attention on the patient while receiving navigation guidance.
Solution Approach 2:
The patent transitions from 2D display monitors to 3D projection mapping onto the patient's anatomical surfaces. This dimensional change allows surgical navigation information to be displayed in the same spatial context as the surgical field, eliminating the need for attention shifting between separate display planes.
2Measurement precision
If fiducial markers are placed on the patient or instrument for tracking, then tracking accuracy is improved, but the procedure complexity and time increase
Solution Approach 1:
The patent extracts and eliminates the fiducial markers from the surgical navigation system. Instead of requiring physical markers to be placed on the patient or instruments, the system uses markerless tracking technology that identifies and tracks anatomical landmarks and instrument features directly through imaging, thereby maintaining tracking accuracy while removing the complexity of marker placement.
Solution Approach 2:
The system enables the patient's anatomy and surgical instruments to serve their own tracking function without external markers. Natural anatomical features and instrument geometries are used as self-identifying markers, allowing the system to perform tracking using the objects themselves rather than requiring additional marker components.
3Reliability
If fiducial markers are used for tracking, then registration can be achieved, but the surgical field is obscured and workflow is interrupted
Solution Approach 1:
The patent removes fiducial markers from the surgical field entirely, replacing them with markerless registration techniques. This extraction eliminates the visual obstruction and workflow interruptions caused by marker placement and identification, while maintaining registration reliability through alternative computer vision and image recognition methods.
Solution Approach 2:
The patent replaces the mechanical marker-based registration system with an optical and computational system. Instead of physically placing and detecting markers, the system uses imaging sensors, computer vision algorithms, and image processing to achieve registration, thereby eliminating the mechanical interference in the surgical field.
4Ease of manufacture
If 2D displays are used for surgical guidance, then the system is easy to implement, but spatial understanding of anatomy is limited
Solution Approach 1:
The patent transforms 2D display output into 3D projection mapping that conforms to the patient's anatomical surfaces. This dimensional transformation preserves and enhances spatial anatomy information by displaying surgical navigation data in the actual three-dimensional context of the patient's body, providing intuitive spatial understanding while maintaining implementation feasibility.
Solution Approach 2:
The patent applies projection mapping that adapts to local anatomical surfaces, allowing surgical information to be displayed with appropriate spatial orientation and scale at each location on the patient's body. This local adaptation preserves detailed spatial anatomy information that would be lost in a uniform 2D display, while the underlying technology remains implementable with standard projection equipment.
Data Source
AI summary
Some embodiments described herein relate to a method (e.g., a computer-implemented method) that includes receiving data associated with an operative field that includes a subject from an optical sensor. A three-dimensional (3D) virtual model associated with at least one of the subject or an object in the operative field can be accessed, and an observed mesh that includes a representation of the subject, based on the data received from the optical sensor can be defined. A virtual 3D environment, including the virtual model can be defined. The virtual model can be registered to the observed mesh, or the observed mesh can be registered to virtual model. A rendering of the virtual model can be projected, in real time, into the operative field such that the rendering of the virtual model is scaled and oriented relative to the at least one of the subject or the object in the real-world operative field as it appears in the virtual 3D environment.