Touchless Gesture Interface for Task Automation on Computer Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional human-machine interfaces in portable computers, such as keyboards and trackpads, have remained unchanged for decades, limiting their use cases and failing to adapt to emerging trends like 3D content creation, virtual reality, and artificial intelligence, relying heavily on typing-based interactions.
Innovation Solution
A touchless gesture interaction method using a touchless gesture input system, machine-learning models, and sensor arrays to detect and interpret user gestures, enabling task automation and seamless interaction without physical contact.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional typing-based interfaces with physical input devices are used, then device simplicity and reliability are maintained, but adaptability to emerging use cases and user productivity are limited
Solution Approach 1:
The patent replaces mechanical input devices (keyboard, trackpad) with a camera-based vision system that captures hand gestures and translates them into digital commands. This substitution eliminates the need for physical contact while enabling diverse interaction modes for emerging use cases like 3D content creation and virtual reality.
Solution Approach 2:
The vision-based interface serves multiple functions: it detects hand gestures for navigation, recognizes objects for contextual actions, enables touchless interaction for hygiene scenarios, and supports both 2D and 3D interaction paradigms. This multi-functionality addresses various emerging use cases without requiring separate specialized devices.
2Ease of operation
If touchless gesture input system is implemented, then ease of operation and accessibility are improved, but device complexity and manufacturing cost increase
Solution Approach 1:
The system uses the user's own hand as the input interface, eliminating the need to learn complex gesture syntax or adapt to unfamiliar input methods. The natural hand movements are directly translated into meaningful commands, making the interface intuitive and self-explanatory while improving accessibility for users with physical disabilities.
3Productivity
If machine-learning model is used for task automation, then productivity is increased, but processing time and energy consumption increase
Solution Approach 1:
The system performs preliminary analysis by continuously monitoring hand gestures and pre-processing visual data in real-time. When a gesture pattern is recognized, the corresponding task automation template is already prepared and can be executed immediately, minimizing processing delays while maintaining high productivity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
According to a first aspect of the present invention, it may be provided a touchless gesture interaction method for controlling a computer device. The method may be computer-implemented. The method may comprise providing, using a display means and at least one user input system including a touchless gesture input system, a user interface for receiving user input. The touchless gesture input system may be configured to generate control commands based on received touchless user input. The control commands may be configured for controlling the computer device. The method may comprise receiving, by the computer device, an indication indicating that a user intends to perform a task on the computer device. Performing the task may involve a sequence of steps to be performed by the computer device based on user input. The method may comprise, in response to the computer device being operated in a task training mode, obtaining, using the at least one user input system, user input configured to causing the computer device to perform at least part of the sequence of steps. The method may comprise generating, using a machine-learning model, a task automation template based on the obtained user input, wherein the task automation template is configured to emulate user input causing the computer device to complete the task and/or to perform the task at a later point in time without operating the computer device in the task training mode.