Multimodal Formula Input via Gesture and Speech Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Entering complex technical formulas into computing devices is cumbersome due to the diversity and complexity of symbols, which do not readily translate to standard computer input devices, making it impractical for users to input and verify mathematical and chemical equations effectively.
Innovation Solution
A method and system that utilize multimodal grammars to allow input through various modalities (GUI, text, voice, gestures) and provide visual feedback, enabling users to enter and display technical formulas using different input methods interchangeably, with automatic conversion between modalities and error checking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If standard computer input devices (keyboard, touch screen) are used to enter technical formulas, then the input process becomes cumbersome and impractical, but using specialized input devices would increase device complexity
Solution Approach 1:
The patent applies universality by enabling standard computer input devices to handle multiple input modalities (text, handwriting, speech, gestures) for entering technical formulas. The system allows users to input formulas through familiar devices like keyboards and touch screens while automatically recognizing and processing different input types, making the device versatile without increasing physical complexity
Solution Approach 2:
The patent introduces an intermediary processing layer that translates various input modalities into standardized technical formula representations. This mediator component handles the conversion between different input methods (handwriting recognition, speech-to-text, gesture interpretation) and the target formula format, shielding users from complexity while maintaining ease of operation
2Adaptability or versatility
If the full range of technical symbols is made available for input, then the input process becomes too wide and diverse to fit into standard computer input devices, but limiting symbols would reduce the ability to express complex equations
Solution Approach 1:
The system achieves universality by supporting multiple input modalities (text entry, handwriting recognition, speech input, gesture control) that can all represent the same mathematical or technical concepts. This allows the full range of technical symbols to be accessible through various methods rather than requiring specialized physical keys for each symbol
Solution Approach 2:
The patent applies dimensionality change by transitioning from a one-dimensional keyboard layout to multi-dimensional input methods including spatial gestures, temporal speech patterns, and graphical handwriting. This allows access to a broader symbol set by utilizing additional input dimensions beyond traditional key presses
3Measurement precision
If complex technical formulas are entered using written language equivalents, then the accuracy and precision of the formula representation decreases, but using symbolic notation increases the difficulty of entry
Solution Approach 1:
The system uses an intermediary recognition and translation layer that converts ease-of-entry inputs (handwriting, speech, gestures) into precise symbolic representations. For example, handwriting recognition technology mediates between the user's natural writing and the precise mathematical symbols, maintaining accuracy while reducing entry difficulty
Solution Approach 2:
The patent implements feedback mechanisms that provide real-time verification and correction during formula entry. As users input formulas through various modalities, the system provides visual feedback showing the interpreted formula, allowing users to verify accuracy and make corrections before final submission
Data Source
AI summary
The claimed subject matter provides a method for entering technical formulas. The method includes receiving a sequence of inputs on a computing device. The sequence of inputs specifies the technical formula. The method also includes displaying the technical formula in response to receiving the sequence of inputs. Additionally, the method includes determining an alternate sequence of inputs that specify the technical formula. The alternate sequence of inputs may be input to the computing device to display the technical formula. The method further includes displaying the alternate sequence.