Online Handwriting Recognition with Multitask Motion Sensing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing online handwriting recognition systems are costly due to the use of multiple sensors and require complex neural network architectures, and they do not perform continuous recognition effectively.
Innovation Solution
A system utilizing a handwriting instrument with a motion sensor and a machine learning model trained in a multitask manner to recognize online handwriting, incorporating a shared backbone neural network for stroke segmentation and character classification, enabling efficient and continuous recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If three sensors (accelerometer, gyroscope, magnetometer) are used for online handwriting recognition, then recognition capability is improved, but device cost increases
Solution Approach 1:
The patent extracts and removes the magnetometer sensor from the sensing system, keeping only the accelerometer and gyroscope. This extraction principle reduces device cost while maintaining recognition capability through optimized use of the remaining sensors combined with neural network processing
Solution Approach 2:
The accelerometer and gyroscope sensors are made to serve multiple functions: they detect both stroke position and motion characteristics, and their data is processed through neural networks that perform multiple tasks including character recognition and gesture identification, replacing the need for the magnetometer
2Measurement precision
If a neural network trained with a dictionary of predetermined words is used, then word recognition is improved, but continuous recognition capability deteriorates
Solution Approach 1:
The system dynamically adapts between two operational modes: dictionary-based recognition mode for predefined words and continuous character recognition mode for arbitrary text. The neural network processes data continuously and can switch between recognition strategies based on the input pattern, enabling both high accuracy for known words and flexibility for unknown text
Solution Approach 2:
The system changes its processing parameters by adjusting the neural network configuration and data analysis approach depending on whether the input matches dictionary patterns or requires continuous character-by-character recognition, allowing optimal performance for both scenarios
3Measurement precision
If four neural networks are used for online handwriting recognition, then recognition accuracy is improved, but system complexity increases
Solution Approach 1:
The patent merges multiple neural network functions into a single integrated neural network architecture. Instead of using four separate neural networks for different processing stages, the system employs one unified neural network that performs feature extraction, pattern recognition, and classification simultaneously, reducing system complexity while maintaining accuracy
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system achieves efficient and continuous online handwriting recognition by effectively segmenting strokes and classifying characters using a single motion sensor and a multitask trained model, capable of recognizing characters in any language and handling in-air movements.
Implementation Method 1
a module comprising at least one motion sensor configured to acquire motion data on the handwriting of a user when the user is writing a characters sequence with the handwriting instrument
Data Source
AI summary
This invention concerns a system for recognizing online handwriting comprising: a handwriting instrument including a body extending longitudinally between a first end and a second end, the first end having a writing tip which is able to write on a support, the handwriting instrument further including a module comprising at least one motion sensor configured to acquire motion data on the handwriting of a user when the user is writing a characters sequence with the handwriting instrument, a calculating unit communicating with the at least one motion sensor and configured to analyze the motion data by a machine learning model trained in a multitask way such that it is capable of performing at least two tasks at the same time, the machine learning model being configured to deliver as an output the characters sequence which was written by the user with the handwriting instrument.