Touch Screen Gesture Classification via Neural Network Image Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current touch screen technologies face challenges in accurately classifying complex user inputs, such as gestures, especially when they involve varying pressures, as existing systems struggle to distinguish between static and dynamic interactions effectively.
Innovation Solution
A sensing array with pressure-sensitive materials, like QTC®, is integrated into touch screens, providing both positional and extent data, which is converted into image data and processed by an artificial neural network to classify mechanical interactions, including gestures with varying pressures, enhancing security and input recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional touch screen pressure sensing is used, then simple pressure detection is achieved, but complex gesture classification accuracy deteriorates
Solution Approach 1:
The patent transforms pressure sensing data from traditional spatial coordinates into an image domain representation. By converting pressure magnitude and distribution into visual image data, the system enables neural networks to process tactile information more effectively, significantly improving gesture classification accuracy while maintaining manageable system complexity.
Solution Approach 2:
The patent replaces traditional mechanical pressure threshold detection with an artificial neural network-based classification system. This substitution allows the system to interpret complex pressure patterns and gestures that go beyond simple binary detection, enabling accurate classification of multiple gesture types including static and dynamic interactions.
2Loss of information
If pressure-sensitive materials like QTC® are integrated, then positional and extent data are provided, but system complexity increases
Solution Approach 1:
The patent integrates pressure-sensitive materials that simultaneously provide both positional information and pressure magnitude data through a single sensing array. This multi-functional approach ensures complete pressure data capture without requiring separate sensing systems, thereby limiting the increase in overall system complexity.
Solution Approach 2:
The patent introduces an intermediary image conversion process that translates complex pressure array data into a standardized image format. This intermediary representation simplifies the processing of complete pressure information by neural networks, making the system more manageable despite the comprehensive data collection.
3Measurement precision
If neural networks process image data, then gesture recognition accuracy improves, but processing time increases
Solution Approach 1:
The patent performs preliminary conversion of pressure data into image format before neural network processing. This pre-processing step organizes the data in a way that is optimized for neural network input, reducing the computational burden during actual gesture recognition and thereby minimizing processing time while maintaining high accuracy.
4Adaptability or versatility
If static and dynamic gestures are classified together, then system versatility improves, but classification difficulty increases
Solution Approach 1:
The patent employs a dynamic neural network classification system that can adapt to both static and dynamic gesture patterns. The system processes pressure data over time sequences, enabling it to differentiate between stationary presses and moving gestures by analyzing temporal changes in the pressure distribution, thus handling diverse gesture types uniformly.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enables precise classification of user inputs, including complex gestures, by converting pressure data into image data that can be recognized by neural networks, improving security and input accuracy, even when inputs are dynamic or involve multiple pressures.
Implementation Method 1
each said sensing element comprises a pressure sensitive material which is responsive to an applied pressure
Data Source
AI summary
A method of classifying a mechanical interaction on a sensing array is described. The sensing array comprises a plurality of sensing elements and the method comprises the steps of identifying positional (x,y) and extent (z) data in response to a mechanical interaction such as a finger press in the sensing array; converting the positional and extent data to image data to produce an image; and classifying the positional and extent data by providing the image to an artificial neural network.