Touch Gesture Stroke Recognition Using BLSTM Sub-Stroke Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gesture recognition techniques in touch-based user interfaces rely on manual definition of heuristics, which perform poorly for elaborate gestures and are difficult to update, and lack flexibility in recognizing new gesture strokes.
Innovation Solution
A method using a neural network-based stroke classifier that segments ink points into sub-strokes, generates feature vectors, and applies them to a trained recurrent BLSTM network for accurate gesture recognition, with timestamp correction and resampling to improve performance across devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If rules-based gesture recognition techniques are used, then the system is easier to implement, but the performance deteriorates for elaborate and atypical gesture strokes
Solution Approach 1:
The patent replaces the mechanical/rules-based gesture recognition system with a neural network-based system. The neural network learns gesture patterns from training data and can recognize elaborate and atypical gestures that rule-based systems cannot handle, thereby improving recognition performance while maintaining implementation feasibility through standardized neural network frameworks.
2Device complexity
If rules-based gesture recognition techniques are used, then the system structure is simpler, but the adaptability to new gesture strokes deteriorates
Solution Approach 1:
The patent employs a neural network model where gesture recognition is achieved through parameter learning rather than fixed rules. When new gesture strokes need to be added, the system requires only retraining the neural network with new data, which changes the internal parameters of the network. This approach maintains relatively simple system structure while providing high adaptability to new gestures.
3Measurement precision
If sub-stroke segmentation is applied, then the recognition accuracy improves by maintaining path information, but the computation time increases
Solution Approach 1:
The patent applies segmentation by dividing the gesture stroke into multiple sub-strokes based on temporal or spatial criteria. Each sub-stroke is processed independently to extract features, and then the segmented features are fed into the neural network for classification. This segmentation strategy improves recognition accuracy by preserving path information while the computational overhead is managed through efficient segmentation algorithms and optimized neural network processing.
Data Source
AI summary
A method for recognizing gesture strokes in user input, comprising: receiving data generated based on the user input, the data representing a stroke and comprising a plurality of ink points in a rectangular coordinate space and a plurality of timestamps associated respectively with the plurality of ink points; segmenting the plurality of ink points into a plurality of segments each corresponding to a respective sub-stroke of the stroke and comprising a respective subset of the plurality of ink points; generating a plurality of feature vectors based respectively on the plurality of segments; and applying the plurality of feature vectors as an input sequence representing the stroke to a trained stroke classifier to generate a vector of probabilities including a probability that the stroke is a non-gesture stroke and a probability that the stroke is a given gesture stroke of a set of gesture strokes.