Smart Pen Gesture Recognition for Symbol Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for teaching writing and recognition of symbols, such as Mandarin Chinese characters or mathematical formulas, lack immediate and meaningful feedback, making it difficult for students to learn proper stroke order and syntax, and are inefficient due to the need to constantly switch focus between reference materials and writing surfaces.
Innovation Solution
A pen-based computing system using a smart pen and dot-enabled paper that captures and compares user gestures to target gestures, providing feedback and determining the next instruction, allowing for the creation of instructional modules that guide users through the writing process with visual, audio, and haptic feedback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If a textbook or printed material is used for teaching symbol writing, then the user can access exemplars and instructions, but the user must constantly shift visual focus between the reference material and the writing surface, reducing learning efficiency
Solution Approach 1:
The patent combines the exemplar display and writing surface into a single integrated display area. The system displays the target symbol, stroke order instructions, and accepts user input all in one location, eliminating the need to shift focus between separate textbooks and writing surfaces. This merging of functions directly addresses the visual focus switching problem while maintaining full contextual information.
2Loss of time
If a workbook is used to allow writing near the exemplar, then the locus of visual focus is narrowed, but the workbook is consumable, expensive, and offers no dynamic or contextual feedback
Solution Approach 1:
The system provides real-time feedback by comparing the user's drawn symbol with the target symbol and evaluating stroke order correctness. The computer analyzes the input gestures, determines whether they match the required stroke sequence, and provides immediate corrective feedback. This dynamic feedback capability replaces the static workbook approach while adding adaptive instructional value.
Solution Approach 2:
The system transitions from the static, consumable nature of workbooks to a dynamic, reusable digital platform. The display can update instructions, show different symbols, provide real-time feedback, and adapt to user performance. This dynamic capability allows the same resource to be used indefinitely with varying instructional content and feedback based on individual student needs.
3Adaptability or versatility
If tablet PCs are used to provide dynamic visuals and immediate feedback, then contextual interpretation and multi-modal feedback are available, but the devices are expensive, bulky, consume substantial power, and provide unfamiliar tactile feedback
Solution Approach 1:
The patent uses a simple optical scanner or camera as an intermediary to capture the user's writing on conventional paper. This intermediary device bridges the gap between the familiar, low-cost paper-based writing experience and the digital processing capabilities of the computer. The system scans the paper, recognizes the handwriting, and provides digital feedback without requiring the user to write directly on a complex tablet interface.
Solution Approach 2:
Instead of requiring users to write directly on expensive tablet screens, the system creates a digital copy of the user's paper-based writing through scanning or imaging. This copying approach allows the system to process and provide feedback on the user's natural paper writing experience, avoiding the need for users to adapt to unfamiliar electronic writing surfaces while still delivering digital feedback capabilities.
4Device complexity
If traditional instruction methods are used without immediate feedback, then instructional materials can be simple, but students lack direct feedback and encouragement regarding their success in accurately recreating symbols
Solution Approach 1:
The system implements automated feedback by comparing the user's drawn symbol against the target symbol and evaluating stroke order accuracy. The computer provides immediate feedback on whether the symbol was drawn correctly and whether the stroke sequence matched the required pattern. This automated feedback mechanism delivers reliable learning guidance without requiring complex human instruction for each student.
Data Source
AI summary
An instruction is presented to a user for making a target gesture. The target gesture may be a portion of an exemplary or model symbol. The instruction may be presented in various ways, such as being printed on a writing surface or being played in audio format through a speaker of a smart pen device. A writing gesture made on a writing surface by the user is digitally captured using a smart pen device. The captured writing gesture is compared to the target gesture and feedback is determined based on this comparison. This feedback may indicate the correctness of the user's writing gesture. The feedback may be presented to the user through various means, including through the smart pen device. The comparison may also be used to determine a next instruction for the user to follow.