Virtual Keyboard Authentication Using AR Gestures and Tactile Input
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing authentication methods, such as multi-factor authentication based on numerical inputs, are prone to observational attacks like shoulder surfing, smudge attacks, and thermal attacks, and are not adaptable to various devices like laptops and smartphones.
Innovation Solution
A method and system that allows users to configure multiple keypad configurations on electronic devices, including laptops and smartphones, using tactile force and augmented reality (AR) gestures, enabling customizable virtual keyboards with alphanumeric characters, which are resistant to observational attacks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multi-factor authentication based on numerical inputs is used, then authentication security is improved, but vulnerability to observational attacks (shoulder surfing, smudge attacks, thermal attacks) increases
Solution Approach 1:
The patent introduces an intermediary layer between the user and the authentication system by using AR gestures and tactile interactions as mediators. Instead of directly entering numerical PINs that can be observed, the system uses gesture recognition and tactile feedback as intermediate steps that convey user intent without exposing sensitive information to observers.
Solution Approach 2:
The patent replaces traditional mechanical input methods (physical keypad entry, touchscreen typing) with augmented reality gesture recognition and tactile force detection. This substitution eliminates the visual exposure inherent in conventional input methods while maintaining authentication functionality through spatial gesture patterns and force-sensitive detection.
2Object-affected harmful factors
If haptic lock techniques are used for discreet PIN entry, then resistance to shoulder surfing is improved, but user confusion increases in scenarios requiring large number input
Solution Approach 1:
The patent transitions from one-dimensional linear input (typing digits sequentially) to three-dimensional spatial gesture recognition. Users define authentication patterns by moving through 3D space, creating geometric shapes or spatial sequences that are inherently more distinguishable and less prone to confusion, while AR visualization provides clear feedback about the gesture being formed.
Solution Approach 2:
The system dynamically adapts the authentication interface based on the device type and context. For devices with trackpads, it uses 2D gesture patterns; for smartphones, it employs 3D spatial gestures. The AR overlay dynamically updates to show real-time gesture recognition status, providing continuous feedback that reduces user confusion.
3Object-affected harmful factors
If illusion PIN entries with hybrid images are used, then shoulder surfing protection is improved, but applicability to laptops and trackpad devices is lost
Solution Approach 1:
The patent creates a universal authentication system that functions across multiple device types (smartphones, laptops, tablets, AR devices) by using gesture recognition and tactile detection that are independent of specific hardware interfaces. The core authentication logic remains the same while adapting to different input capabilities of each device category.
Solution Approach 2:
The system changes the fundamental parameters of interaction from visual-display-dependent (illusion PIN requiring image rendering) to sensor-dependent (gesture recognition using accelerometer, gyroscope, touch sensors, and force detection). This parameter change enables the same authentication mechanism to work across diverse device types with different display and input capabilities.
4Ease of operation
If traditional PIN entry methods are used, then ease of input is improved, but vulnerability to smudge attacks and thermal attacks increases
Solution Approach 1:
The patent extracts the sensitive authentication input from the visual domain where smudge and thermal attacks can occur. By moving authentication to the gesture and tactile domains through AR and force-sensitive detection, the system removes the vulnerable intermediate step of visual display and physical contact with sensitive information.
Data Source
AI summary
Embodiments relate to a method and an electronic device for modifying a user interface of the electronic device. The method includes detecting, on the electronic device, a selection of a multi-keypad configuration among a plurality of multi-keypad configurations. Based on the selection of the multi-keypad configuration, a plurality of virtual keys are displayed on the user interface. The method includes receiving, on the user interface, a user touch gestures to configure one or more of the plurality of virtual keys to generate a virtual keyboard. The method includes updating machine learning models, image classifier models, and memory of the electronic device to update and store the generated virtual keyboard including the configuration of the one or more of the plurality of virtual keys to initiate user authentication in real time.