Surgical Instrument Pose Estimation Using Key Point Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer-assisted surgical systems face challenges in accurately identifying and tracking surgical instruments in complex surgical environments with varying lighting, obstructions, and orientations, and in determining how these instruments interact with anatomy during procedures.
Innovation Solution
A system utilizing machine learning models to autonomously identify key points of surgical instruments, group them, and determine their poses in real-time using video data from endoscopic and external cameras, with machine learning models trained on annotated surgical data to improve precision and consistency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional computer vision methods are used to identify surgical instruments, then the system complexity is lower, but the measurement precision and reliability deteriorate in complex surgical environments with varying lighting, obstructions, and orientations
Solution Approach 1:
The patent replaces traditional computer vision algorithms with machine learning models that autonomously identify key points and determine poses of surgical instruments. This substitution enables the system to handle complex surgical environments with varying lighting, obstructions, and orientations, significantly improving measurement precision while the modular architecture keeps system complexity manageable
Solution Approach 2:
The patent changes the operational parameters of the identification system by using trained machine learning models that can adapt to different lighting conditions, obstructions, and instrument orientations. This allows the system to maintain high accuracy across varying surgical environments without requiring manual parameter adjustments
2Productivity
If multiple surgical instruments are tracked simultaneously, then the productivity and surgical guidance quality improve, but the difficulty of detecting and measuring increases due to complex interactions and occlusions
Solution Approach 1:
The patent segments the tracking problem by identifying and tracking key points on each surgical instrument independently. The machine learning model detects specific anatomical landmarks and instrument features separately, then integrates this information to determine the pose and interaction of multiple instruments simultaneously, reducing the overall detection difficulty
Solution Approach 2:
The patent introduces key point detection as an intermediary step between raw image data and final instrument pose estimation. By first identifying characteristic points on instruments and using these as intermediaries to infer instrument states and interactions, the system simplifies the tracking of multiple instruments in complex scenes
3Reliability
If real-time pose estimation is performed for multiple instruments, then the surgical feedback quality improves, but the use of energy and computational resources increases
Solution Approach 1:
The patent performs preliminary action by pre-training machine learning models on extensive surgical datasets before deployment. This offline training phase allows the models to learn instrument characteristics and surgical patterns in advance, enabling efficient real-time pose estimation during surgery with reduced computational resource consumption
Solution Approach 2:
The patent applies partial action by focusing computational resources on detecting and tracking only the key points and critical features of surgical instruments rather than processing entire images. This selective approach maintains high reliability for surgical feedback while significantly reducing real-time computational energy consumption
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Techniques are described for improving computer-assisted surgical (CAS) systems. A CAS system includes endoscopic cameras that provide video stream of a surgical procedure. The CAS system also includes surgical instruments to perform one or more surgical actions. According to one or more aspects key points of the surgical instruments, such as tips, joints, etc., are automatically detected in the video stream. The detected key points are used to determine poses of the surgical instruments. In some aspects, detection of the instruments and estimation of poses of the respective instruments are performed concurrently.