Binocular AR Marker Localization for Mobile Soft Tissue Biopsy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current marker localization systems for soft tissue tumors, particularly in breast surgery, lack inter-marker connectivity, create MRI artifacts, and fail to provide accurate real-time visualization of markers and tumor regions, especially in mobile tissue environments where tumors are no longer visible due to preoperative therapy, necessitating improved augmented reality systems for precise biopsy and surgery.
Innovation Solution
A marker localization system integrated with binocular AR glasses and a processing device that allows real-time intraoperative coordinate framing of markers and ROI using smart markers, enabling precise geometrical alignment and visualization without preoperative imaging fusion, utilizing passive and activated markers with a locator for accurate marker placement and triangulation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional marker localization systems are used, then markers can be localized, but they lack inter-marker connectivity and cannot provide accurate visualization of tumor edges
Solution Approach 1:
The patent combines multiple discrete marker localization functions into a unified AR system that simultaneously tracks multiple markers, maintains inter-marker connectivity through spatial relationships, and overlays this information on real-time tissue visualization. The AR glasses integrate marker positions, tumor boundaries, and tissue structure into a single augmented view, resolving the information loss problem while maintaining localization accuracy.
Solution Approach 2:
The patent introduces AR glasses as an intermediary between the surgical field and the surgeon's perception. This intermediary layer overlays virtual marker connections, tumor boundary definitions, and spatial relationships directly onto the physical tissue view, enabling accurate visualization without requiring separate discrete localization events for each marker.
2Loss of information
If preoperative imaging fusion is used, then tumor location can be visualized, but it cannot provide real-time updates in mobile tissue environments
Solution Approach 1:
The patent transitions from static preoperative imaging to dynamic real-time AR visualization. The system continuously updates marker positions and tumor boundary definitions as tissue moves during surgery, maintaining accurate spatial relationships in real-time. The AR overlay dynamically adapts to tissue displacement, ensuring reliability in mobile tissue environments like the breast.
Solution Approach 2:
The system incorporates real-time feedback from intraoperative localization events, continuously adjusting the AR visualization to match current tissue positions. This feedback loop ensures that tumor location information remains accurate and up-to-date throughout the surgical procedure, adapting to any tissue movement or deformation.
3Measurement precision
If multiple discrete markers are placed to define tumor margins, then localization can be achieved, but the system complexity increases and inter-marker connectivity is lost
Solution Approach 1:
The patent merges multiple discrete marker tracking functions into a single integrated AR system that simultaneously manages all markers, maintains their spatial relationships, and overlays connectivity information. This unified approach reduces operational complexity while achieving precise tumor margin localization through automated spatial relationship maintenance.
Solution Approach 2:
The system creates a virtual copy of the physical marker arrangement in the AR overlay, maintaining spatial relationships and connectivity without requiring complex physical linkages between markers. The virtual representation automatically reflects the actual marker positions and their relationships, simplifying the physical system while maintaining precision.
4Adaptability or versatility
If MRI-compatible markers are used, then imaging can be performed, but they create significant MRI artifacts
Solution Approach 1:
The patent extracts the localization function from the imaging process. Instead of using markers that interact with MRI fields (creating artifacts), the system uses passive optical markers visible to AR cameras. The localization and imaging functions are separated: optical markers provide localization for AR visualization, while separate MRI sequences capture tissue information without marker interference.
Solution Approach 2:
The patent introduces AR glasses as an intermediary that replaces the need for MRI-compatible markers. The AR system processes images from optical cameras to track marker positions and overlay tumor information, eliminating the need for markers that would create MRI artifacts while maintaining full imaging compatibility.
Data Source
AI summary
The combination of a marker system with binocular AR glasses to create a soft tissue procedure system used for surgery, biopsies, etc. A marker localization system is functionally integrated with augmented reality (AR) glasses. This soft tissue system can be applicable for AR surgeries in soft tissues involving mobile and static anatomies anywhere in the body, such as for example, breast, soft tissue, lungs, lymph nodes, liver surgeries, etc. By the placement of single or multiple markers and localizing the markers with a locator, such as a hand-held locator as a non-limiting example, the above-mentioned system provides a real-time intraoperative coordinate frame for the virtual projection of the markers (and the associated ROI) on/in the surgical field/biopsy field of view, using commercial off-the-shelf computer hardware (laptops, tablets, etc.) for the required image processing.