Deep Learning User Switching Detection in Robotic Surgery

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing robotic surgical systems lack mechanisms to detect user-switching events during surgical procedures, leading to inaccurate eye-tracking control signals and unsafe robotic arm or tool motions.

Innovation Solution

The implementation of a user-presence/absence recognition system using deep learning and convolutional neural networks (CNNs) to detect user-switching events by analyzing video images from the user-seating area of the surgeon console, prompting the new user for identification, and triggering recalibration of console settings if necessary.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If the robotic surgical system uses existing eye tracking mechanisms without user switching detection, then the system maintains continuous operation, but the eye tracking settings become inaccurate when a new surgeon uses the console

Engineering Contradiction:

Improveeye tracking control signal accuracyVSAvoidsystem structure

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The system performs preliminary detection of user presence and identity before allowing surgical operation to proceed. The CNN-based image recognition model continuously monitors the console area to identify when a new surgeon approaches or takes the seat, triggering a recalibration sequence before inaccurate eye tracking can occur.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements a feedback loop where the CNN model continuously analyzes images from the console area, detects user presence and identity changes, and automatically triggers recalibration procedures. This closed-loop feedback ensures eye tracking settings remain accurate by detecting and responding to user switching events in real-time.

Inventive Principle:

Principle #23Feedback

2Measurement precision

If the system implements user switching detection using deep learning and CNNs, then user switching events are accurately detected and settings are recalibrated, but the device complexity and computational requirements increase

Engineering Contradiction:

Improveuser switching detection accuracyVSAvoidsystem structure

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system introduces a CNN-based image recognition model as an intermediary component that specializes in user detection and identification. This modular approach allows the complex deep learning functionality to be separated from the core surgical control system, enabling accurate user switching detection while maintaining clear system architecture and manageable complexity.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Reliability

If the system continuously monitors the user seating area for user switching events, then user safety is improved through timely recalibration, but the computational energy consumption increases

Engineering Contradiction:

Improvesurgical procedure safetyVSAvoidcomputational energy consumption

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The system uses periodic action by triggering the CNN-based user detection and recalibration process only at appropriate intervals and conditions - specifically when user switching events are detected or at predetermined checkpoints during the surgical procedure. This reduces continuous computational overhead while maintaining safety through timely recalibration when needed.

Inventive Principle:

Principle #19Periodic action

Data Source

PatentUS20250054302A1User switching detection during robotic surgeries using deep learning

Publication Date: 2025.02.13 AURIS HEALTH INC

AI summary

Disclosed are various user-presence/absence detection techniques based on deep learning. These user-presence/absence detection techniques can include building/training a deep-learning model including a user-presence/absence classifier based on training images of a user-seating area of a surgeon console under various clinically-relevant conditions. The trained user-presence/absence classifier can then be used during teleoperation/surgical procedures to monitor/track users in the user-seating area of the surgeon console, and continuously classify captured real-time video images of the user-seating area into either a user-presence classification or a user-absence classification. In some embodiments, the disclosed techniques can be used to detect a user-switching event at the surgeon console when a second user is detected to have entered the user-seating area after a first user is detected to have exited the user-seating area. If the second user is identified as a new user, the disclosed techniques can trigger a recalibration procedure to recalibrate surgeon-console settings for the new user.