Robotic Surgery Tool Segmentation Using Kinematic Search Areas
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current robotic surgery systems face challenges in efficiently identifying and segmenting surgery tools within image data due to the limited field of view and computational resource-intensive processes.
Innovation Solution
A computer-implemented method that utilizes kinematic data and system property data to determine the nominal position of a robotic surgery arm, thereby narrowing the search area within image data and improving the efficiency of surgery tool segmentation using object recognition techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the entire image data is processed using object detection techniques to identify surgery tools, then the identification accuracy is improved, but the computational burden and processing time increase significantly
Solution Approach 1:
The patent divides the image processing task into two segments: first, a narrow search area is defined based on robot arm kinematics to locate the surgery tool, and then object detection is applied only within this segmented region. This segmentation approach maintains identification accuracy while reducing the computational burden of processing the entire image.
Solution Approach 2:
The patent performs preliminary action by using robot arm kinematic data to pre-determine the expected location and define a narrow search area before applying object detection techniques. This preliminary localization step reduces the search space, allowing object detection to focus only on relevant regions, thereby improving processing speed without sacrificing accuracy.
2Productivity
If a narrow search area is used to reduce computational burden, then processing speed is improved, but the risk of missing the surgery tool increases
Solution Approach 1:
The patent incorporates feedback by iteratively refining the narrow search area based on object detection results. The search area is initially defined using robot arm kinematics, then object detection is performed within this area, and the results are used to update and refine the search area for subsequent detections. This feedback loop ensures that the narrow search area remains reliable for identifying the surgery tool while maintaining processing speed.
3Adaptability or versatility
If object detection is applied to the entire image data, then comprehensive tool identification is achieved, but the field of view limitation becomes more problematic
Solution Approach 1:
The patent transitions from two-dimensional image space analysis to incorporating third-dimensional robot arm kinematic data. By using robot arm joint angles and link lengths to calculate expected tool positions in 3D space and projecting these onto the 2D image plane, the system defines narrow search areas that compensate for the limited field of view, enabling comprehensive tool identification without processing the entire image.
Data Source
AI summary
Systems and methods for identifying surgery tools of a robotic surgery system in surgery video based on robot kinematics are described. Kinematic data describing a position of a first robotic surgery arm including a surgery tool is used to determine a nominal position of the first robotic surgery arm. A search area within image data obtained during a robotic surgery from an endoscope attached to a second robotic surgery arm is determined based on the kinematic data and a nominal position of the surgery tool. The surgery tool is identified within the search area of the image data using an object detection technique.