Surgical Tool GUI Tracking for Sterile Touchless Input
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Surgeons in sterile environments face challenges interacting with non-sterile computer system interfaces, leading to potential distractions and errors during surgeries due to the need to physically move from the patient to input data, which can result in negative surgical outcomes.
Innovation Solution
A system that allows surgeons to control a graphical user interface using surgical tooling equipment, such as a scalpel or tweezers, by tracking their movements and using image sensors to determine positions and patterns, enabling touchless interaction and reducing the need for physical movement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If surgeons use traditional input devices (foot pedals, surgical assistants, one-time disposals) to interact with computer systems, then the surgeons can input data into the system, but the surgeons must physically move their hand and/or head from the patient to the interface, causing distraction and potential errors
Solution Approach 1:
The patent introduces an intermediary system consisting of image sensors, tracking software, and gesture recognition algorithms that mediate between the surgical tool and the computer interface. This intermediary layer translates surgical tool movements and gestures into GUI commands without requiring the surgeon to physically interact with traditional input devices, thereby eliminating the need to move hand/head from the surgical field while maintaining input accuracy
Solution Approach 2:
The patent replaces mechanical input devices (foot pedals, physical buttons, keyboards) with an optical-mechanical system that uses image sensors to capture surgical tool movements and gesture recognition software to interpret these movements as input commands. This substitution eliminates the physical disconnect between the surgeon's focus area and the input interface, allowing continuous visual attention on the surgical site while maintaining reliable data input
2Measurement precision
If surgeons maintain continuous visual attention on the patient during surgery, then surgical precision is improved, but the ability to interact with computer interfaces is reduced
Solution Approach 1:
The system enables self-service operation where the surgical tool itself becomes the input device. The tracking system automatically captures the tool's position and orientation, and the gesture recognition software automatically interprets these parameters as meaningful commands. This eliminates the need for separate manual input actions, allowing surgeons to maintain visual focus on the patient while the system autonomously captures and processes input data from the surgical tool's natural movements
Solution Approach 2:
The surgical tool serves multiple functions simultaneously: it performs the primary surgical task and also acts as the input device for the computer interface. The same tool that manipulates tissue also provides positional and gestural data for GUI interaction through the tracking system. This multi-functionality eliminates the need for separate input devices and allows continuous visual attention on the surgical site while maintaining full interface interaction capability
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables surgeons to interact with the GUI without leaving their field of view, minimizing distractions and reducing input errors, thereby enhancing surgical precision and safety.
Implementation Method 1
receive a first image from an image sensor; and determine, from the first image and a digital model of surgical tooling equipment, a first position of the surgical tooling equipment within the first image
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
The present disclosure provides a system that may display a graphical user interface (GUI) that includes an icon via a display; may receive a first image from an image sensor; may determine, from the first image and a digital model of surgical tooling equipment, a first position of the surgical tooling equipment within the first image; may display a cursor of the GUI at a second position; may receive a second image from the image sensor; may determine, from the second image and the digital model, a third position of the surgical tooling equipment within the second image; may display the cursor of the GUI at a fourth position; may receive user input that indicates a selection while coordinates of the icon include the fourth position; and may determine that the user input that indicates the selection while the coordinates of the at least one icon include the fourth position.