Robotic Vision Tracking With Virtual Boundaries for Collision Avoidance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional navigation systems for robotic surgery face challenges in tracking and avoiding physical objects near the target site, leading to potential collisions and increased complexity and cost due to the need for additional trackers, and existing object recognition methods are computationally expensive and difficult to implement.
Innovation Solution
A system that integrates a vision device with a robotic device to capture vision data sets from multiple perspectives, allowing for the association of virtual objects with physical objects to define movement constraints, thereby enabling efficient tracking and avoidance of physical objects during surgery without separate, expensive trackers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If separate trackers are added to retractors and other physical objects to enable tracking, then the ability to track physical objects is improved, but the cost and complexity of the navigation system increases substantially
Solution Approach 1:
The patent uses visual copies (images) of physical objects captured by a camera instead of physical trackers. The system captures images of retractors and other physical objects, processes these images to identify object locations and characteristics, and uses this information to track physical objects throughout the surgical procedure. This eliminates the need for expensive physical trackers on each object.
Solution Approach 2:
The patent replaces the mechanical/optical tracking system (physical trackers with light sources and cameras) with a vision-based system using standard cameras and image processing algorithms. The mechanical tracker system is substituted with computational vision techniques including image recognition, feature detection, and computer vision algorithms to achieve the same tracking function.
2Device complexity
If object recognition techniques are used to track physical objects, then the need for separate trackers is eliminated, but the computational cost and difficulty increases
Solution Approach 1:
The patent performs preliminary actions by capturing multiple images of physical objects from different viewpoints during the surgical procedure. These images are processed to create a three-dimensional model or spatial representation of the physical objects before the actual tracking and avoidance operations begin. This preliminary image capture and processing enables efficient real-time tracking without requiring continuous heavy computational processing.
Solution Approach 2:
The patent transforms two-dimensional images captured by the camera into three-dimensional spatial representations of physical objects. By reconstructing the three-dimensional geometry and position of retractors and other objects from multiple camera viewpoints, the system achieves accurate tracking and spatial awareness. This dimensional transformation enables precise object location and movement monitoring with standard imaging hardware.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system for tracking a physical object comprises a robotic device, an instrument attachable to said robotic device, the instrument being configured to treat tissue, a vision device attached to one of said robotic device or said instrument such that said vision device is movable with said robotic device, said vision device configured to generate vision data sets, wherein said vision data sets are captured from multiple perspectives of the physical object enabled by said vision device being moved in a plurality of degrees of freedom when moving said robotic device, and at least one processor being in communication with said vision device, the at least one processor being configured to associate a virtual object with the physical object based on one or more features of the physical object identifiable in said vision data sets, wherein the virtual object at least partially defines a virtual boundary defining a constraint on movement of the robotic device relative to the physical object.