Gesture Stroke Recognition with BLSTM and Timestamp Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gesture recognition techniques in touch-based user interfaces rely on manual definition of heuristics, which perform poorly for elaborate gestures and are difficult to update with new strokes, and are device-dependent due to inconsistent timestamping and sampling.
Innovation Solution
A method using a neural network-based stroke classifier that segments ink points into sub-strokes, generates feature vectors, and corrects timestamps to improve recognition, employing a recurrent BLSTM network to handle temporal dependencies and device inconsistencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If rules-based gesture recognition techniques are used, then the system is simple to implement, but performance deteriorates for elaborate gestures and updates are difficult
Solution Approach 1:
The patent replaces the mechanical/rules-based gesture recognition system with a neural network-based system. The neural network automatically learns gesture patterns from training data, eliminating the need for manual heuristic definition and enabling accurate recognition of elaborate gestures that would be difficult to program with rules.
Solution Approach 2:
The patent changes the fundamental parameter of the recognition system from discrete manual rules to continuous neural network parameters. The neural network adjusts its internal parameters during training to capture complex gesture patterns, allowing flexible adaptation to various gesture types without requiring explicit programming of each gesture variant.
2Device complexity
If rules-based techniques are used, then device complexity is low, but adaptability to new gestures is poor
Solution Approach 1:
The patent employs a neural network whose parameters can be automatically adjusted through retraining on new gesture data. This allows the system to adapt to new gestures by simply providing training examples, without requiring complex reconfiguration or programming of new rules, thus achieving high adaptability while maintaining manageable system complexity.
Solution Approach 2:
The neural network system performs self-learning and self-adjustment through automated training processes. When new gestures need to be recognized, the system can be retrained on new data without human intervention in the core recognition logic, enabling the system to serve itself in adapting to new requirements.
3Ease of manufacture
If manual heuristic definition is used, then implementation is straightforward, but performance deteriorates for atypical gestures
Solution Approach 1:
The patent substitutes manual heuristic definition with a neural network that learns from data. The neural network processes input gestures and automatically adjusts its internal representations to accurately classify gestures, including atypical variants, achieving high measurement precision without the limitations of pre-defined heuristics.
Solution Approach 2:
The neural network system incorporates feedback through training on labeled gesture data. During training, the network receives feedback about correct classifications and adjusts its parameters accordingly, continuously improving its precision in recognizing gestures including atypical ones, whereas manual heuristics have fixed performance without feedback mechanisms.
Data Source
AI summary
A method for recognizing gesture strokes in user input, comprising: receiving data generated based on the user input, the data representing a stroke and comprising a plurality of ink points in a rectangular coordinate space and a plurality of timestamps associated respectively with the plurality of ink points; segmenting the plurality of ink points into a plurality of segments each corresponding to a respective sub-stroke of the stroke and comprising a respective subset of the plurality of ink points; generating a plurality of feature vectors based respectively on the plurality of segments; and applying the plurality of feature vectors as an input sequence representing the stroke to a trained stroke classifier to generate a vector of probabilities including a probability that the stroke is a non-gesture stroke and a probability that the stroke is a given gesture stroke of a set of gesture strokes.