Endoscopic Video Annotation via Voice and Gesture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In minimally invasive surgeries, the reliance on abstract representations of anatomy can lead to communication difficulties when a third party, not familiar with the anatomy, is involved in manipulating medical images, which can increase surgery time and risk of error.
Innovation Solution
A system and method for annotating endoscopic video during procedures using a computer-based interface that allows for voice and gesture commands to overlay graphical annotations onto the video feed, enabling real-time, persistent annotations that can be performed by multiple users, either collocated or remote, using a Kinect-like gesture command input device and voice command input system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a third party not familiar with anatomy manipulates medical images, then the surgeon can focus on the procedure, but communication difficulties arise and error risk increases
Solution Approach 1:
The surgeon performs image manipulation tasks themselves using voice and gesture commands, eliminating the need for assistant intervention. The system responds directly to the surgeon's commands, allowing self-service operation of imaging tools during the procedure.
Solution Approach 2:
Traditional mechanical interaction (mouse, keyboard, touch screen) is replaced with voice commands and gesture recognition. This substitution allows the surgeon to interact with the system hands-free and without requiring translation of physical actions into digital commands by an assistant.
2Productivity
If a third party manipulates medical images, then the surgeon's hands are freed for the procedure, but surgery time increases due to communication overhead
Solution Approach 1:
The surgeon directly controls image manipulation without requiring assistant mediation. Voice commands and gestures enable the surgeon to perform annotations, zoom, pan, and other operations themselves, eliminating the time lost in communicating requests and waiting for execution by another person.
Solution Approach 2:
The surgeon maintains continuous engagement in the surgical procedure while simultaneously controlling imaging functions. The hands-free voice and gesture interface allows uninterrupted workflow, where image manipulation occurs seamlessly during the procedure without breaking the surgical flow or requiring pause for assistant coordination.
3Ease of operation
If traditional input devices are used, then the interface is simple, but the surgeon cannot interact with images during the procedure
Solution Approach 1:
The voice and gesture recognition system serves multiple functions: selecting tools, navigating images, making annotations, and controlling display parameters. This single interface system replaces multiple traditional input devices (mouse, keyboard, touch screen) and enables comprehensive interaction without requiring the surgeon to switch between different input methods or devices.
Data Source
AI summary
The invention relates to annotating live video during endoscopy medical procedures.