Touchless Mobile Device Control via Fingertip Gesture Recognition

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In sterile environments like operating rooms, using touchscreens or keyboards on mobile devices is impractical due to contamination risks, making it difficult for multiple users to control and document procedures effectively.

Innovation Solution

A method and apparatus that utilize a camera on a mobile device to identify fingertip positions in a virtual plane, employing entropy-based image segmentation to allow users to control the device without touching it, enabling documentation of procedures and instrument identification without direct interface interaction.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If a touchscreen or keyboard is used to control the mobile device, then the device can be operated and documented, but the sterile environment is compromised and contamination risk increases

Engineering Contradiction:

Improvedevice controlVSAvoidcontamination risk

Core Design Contradiction:

Ease of operationVSObject-affected harmful factors

Solution Approach 1:

The patent replaces the mechanical touchscreen interface with an optical recognition system. The camera captures images of hand gestures, and image processing algorithms translate these gestures into control commands, eliminating the need for physical contact with the device surface.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent introduces an intermediary layer between the user and the device. Instead of direct touch, users perform gestures in the air, which are then captured and translated by the system through image processing and gesture recognition algorithms.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If a touchscreen interface is used, then single-user operation is supported, but multiple users cannot simultaneously control the device without contamination

Engineering Contradiction:

Improvemulti-user capabilityVSAvoidinfection risk

Core Design Contradiction:

Adaptability or versatilityVSObject-affected harmful factors

Solution Approach 1:

The patent replaces the mechanical touchscreen with an optical gesture recognition system that can identify and track multiple hands simultaneously. The system differentiates between different users based on the spatial distribution and movement patterns of their hand gestures.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Productivity

If direct touchscreen interaction is used, then rapid input is possible, but documentation accuracy and completeness are reduced due to limited user input methods

Engineering Contradiction:

Improvedocumentation speedVSAvoidprocedure documentation completeness

Core Design Contradiction:

ProductivityVSLoss of information

Solution Approach 1:

The patent makes the mobile device multi-functional by integrating camera-based gesture control, voice recognition, and automatic procedure documentation. The system can capture multiple types of user input (gestures, voice commands) and automatically generate comprehensive procedure reports, eliminating the need for manual typing and reducing information loss.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

Enables real-time, accurate, and efficient documentation of procedures in sterile environments by allowing multiple users to control a mobile device without physical contact, maintaining sterility and reducing contamination risks, while also documenting instruments and procedures comprehensively.

Implementation Method 1

identifying a position of a user's fingertips in a virtual plane formed apart from a control interface of a mobile device using a camera of the mobile device and an entropy-based segmentation of an image input from the camera

Methodology Applied

Scientific EffectEntropy-based segmentation:

Data Source

PatentUS8907914B2Methods and apparatus for documenting a procedure

Publication Date: 2014.12.09 GE PRECISION HEALTHCARE LLC

AI summary

Methods and apparatus are disclosed for documenting a procedure in a touchless environment. Example methods disclosed herein include identifying a position of one or more of a user's fingertips using a camera of a mobile device and an entropy-based segmentation of an image input, determining a correspondence between user fingertip position and a control interface of the mobile device, including a plurality of functions to be executed based on user input, triggering the mobile device to execute one or more of the plurality of functions based on the user fingertip position during a procedure, and documenting one or more events of the procedure based on the user input for generating a report to be stored in a storage device of the mobile device.