Bronchoscopy Tool Recognition Using AI Workflow Phase Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current medical procedures lack automated solutions for recognizing tool usage and user behaviors during robotic endoscopic/bronchoscopic procedures, requiring manual post-procedure analysis and no intra-operative mechanism for phase recognition.
Innovation Solution
An AI-based workflow recognition system using machine learning algorithms, particularly deep learning models like CNNs and TCNs, to identify surgical tools and phases from bronchoscopy videos, enabling real-time and post-processing analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual post-procedure analysis is used, then measurement precision can be maintained, but productivity is reduced and loss of time increases
Solution Approach 1:
The patent replaces manual mechanical analysis with an automated AI-based system using deep learning models (CNNs and TCNs) to detect surgical tools and recognize workflow phases from bronchoscopy videos, eliminating the need for manual frame-by-frame review while maintaining high measurement precision
Solution Approach 2:
The system performs self-service by automatically analyzing surgical videos without requiring human intervention for routine detection tasks. The AI model independently processes video data, identifies tools, and recognizes phases, freeing manual resources for more complex tasks
2Productivity
If automated AI-based recognition is implemented, then productivity is improved and loss of time is reduced, but device complexity increases
Solution Approach 1:
The patent segments the complex recognition task into distinct functional components: tool presence detection using CNNs, phase/activity recognition using TCNs, and workflow analysis. This modular segmentation allows each component to be optimized independently while working together as an integrated system
Solution Approach 2:
The AI-based system provides multi-functionality by simultaneously performing tool detection, phase recognition, activity identification, and workflow analysis from a single video input. This universal approach consolidates multiple analysis functions into one system rather than requiring separate specialized systems for each task
3Loss of time
If real-time analysis is performed, then loss of time is reduced, but measurement precision may be compromised due to computational demands
Solution Approach 1:
The system employs periodic action by processing video data in sequential frames and time windows rather than attempting to analyze the entire video stream simultaneously. The TCN processes temporal information in manageable segments, enabling real-time analysis while maintaining precision through systematic frame-by-frame evaluation
Solution Approach 2:
The patent applies preliminary action by pre-processing video data and training the AI models in advance. The CNNs are pre-trained to extract visual features from surgical instruments, and the TCNs are pre-trained to recognize temporal patterns, allowing rapid inference during actual surgical procedures without compromising precision
Data Source
AI summary
A robotic system is configured to automatically identify surgical instruments used during a bronchoscopy procedure. The robotic system can include a video capture device, a robotic manipulator, sensors configured to detect a configuration of the robotic manipulator, and control circuitry communicatively coupled to the robotic manipulator. The control circuitry is configured to perform, using a machine learning classifier, a first analysis of a bronchoscopy video of a patient site to track a medical instrument in the bronchoscopy video. The control circuitry can then identify a set of possible instrument identifications for the medical instrument in the bronchoscopy video based on the first analysis and an identified phase of the bronchoscopy procedure. The control circuitry can then track a motion of the medical instrument in the bronchoscopy video and select an identification from the set of possible instrument identification for the medical instrument based at least on the tracked motion.