Gesture Stroke Recognition via Neural Network Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gesture recognition techniques in touch-based user interfaces are rule-based and perform poorly for complex or atypical gestures, requiring manual updates to add new gesture types, which is inefficient and difficult to implement effectively.
Innovation Solution
A method that segments user input into sub-strokes, generates feature vectors, normalizes them based on document scales, and uses a trained neural network classifier to distinguish between gesture and non-gesture strokes, allowing for easy addition of new gesture types and improved recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If rule-based gesture recognition techniques are used, then the system is simple to implement, but the recognition accuracy deteriorates for complex or atypical gestures
Solution Approach 1:
The patent replaces rule-based gesture recognition with a machine learning-based neural network system. The neural network processes stroke data through multiple layers (embedding layer, LSTM layers, fully connected layers) to automatically learn gesture patterns, substituting manual rule creation with automated computational learning that achieves higher accuracy for complex gestures.
2Device complexity
If manual heuristics are defined for each gesture type, then the recognition can be performed with simple rules, but the difficulty of updating increases when adding new gesture types
Solution Approach 1:
The neural network system performs self-learning through training on gesture data. When new gesture types need to be added, the system automatically adapts by retraining on new data samples, eliminating the need for manual heuristic development. The model self-updates its recognition capabilities through continuous learning from training data.
3Productivity
If the entire stroke is processed as a single sequence, then the computation is straightforward, but the computation time increases and path information is lost
Solution Approach 1:
The patent segments the stroke into multiple sub-strokes based on temporal intervals or spatial criteria. Each sub-stroke is processed independently through the neural network, reducing the computational burden on any single processing unit and enabling parallel processing. This segmentation maintains path information within each segment while improving overall processing efficiency.
4Device complexity
If timestamps are not corrected for device capture issues, then the data processing is simpler, but the recognition accuracy deteriorates due to artifacts
Solution Approach 1:
The patent applies preliminary timestamp correction to the stroke data before feeding it to the neural network. The correction process adjusts timestamps to account for device capture delays or inconsistencies, ensuring that the temporal information used for gesture recognition is accurate. This preprocessing step eliminates artifacts that would otherwise degrade recognition accuracy.
Data Source
AI summary
A method for recognizing gesture strokes in user input applied onto an electronic document, comprising: receiving data based on the user input, the data representing a stroke and comprising a plurality of ink points and a plurality of timestamps associated with the plurality of ink point; segmenting the plurality of ink points into a plurality of segments each corresponding to a respective sub-stroke and comprising a respective subset of the plurality of ink points; determining a scale of the electronic document; generating a plurality of feature vectors based on the plurality of segments; normalizing a subset of the feature vectors according to the scale; and applying the plurality of feature vectors to a trained stroke classifier to generate a vector of probabilities including a probability that the stroke is a non-gesture stroke and a probability that the stroke is a given gesture stroke of a set of gesture types.