Cross-Modality Language Model Updates for Mobile Input Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mobile devices struggle with inefficient learning of new words in both voice and keyboard input methods, leading to unnecessary computations and incorrect autocorrections.
Innovation Solution
A computing system that integrates voice and keyboard input methods by generating an input context data structure to update language models, allowing seamless learning of new words across both modalities, reducing redundant signal analysis and enhancing processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If separate language models are maintained for voice and keyboard input methods, then each modality can be optimized independently, but the device experiences redundant computations and slower learning of new words across modalities
Solution Approach 1:
The patent merges separate language models for voice and keyboard input methods into a unified language model. This consolidation allows the system to learn new words through either modality and apply that knowledge across both input methods, eliminating redundant computations and improving learning efficiency.
Solution Approach 2:
The unified language model serves multiple functions by supporting both voice and keyboard input methods simultaneously. It acts as a universal knowledge base that enhances word recognition and autocorrection across different input modalities, reducing the need for separate processing systems.
2Reliability
If independent language models are used for voice and keyboard IMEs, then each can be optimized for its specific modality, but cross-modality word learning is delayed and less accurate
Solution Approach 1:
The unified language model implements a feedback mechanism where learning from one modality (e.g., voice input) immediately improves performance in the other modality (e.g., keyboard input). When the system learns a new word or correction through either input method, this knowledge is instantly available to both voice and keyboard IMEs, improving autocorrection accuracy and reducing learning time.
3Productivity
If the system processes input signals through separate recognition systems, then each modality can be handled independently, but system bandwidth and processing efficiency are reduced
Solution Approach 1:
The patent combines separate voice and keyboard recognition systems into a unified architecture that shares a common language model. This merging reduces system bandwidth requirements and improves processing efficiency by eliminating redundant processing steps, while the modular design maintains manageable system complexity.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for cross input modality learning in a mobile device are disclosed. In one aspect, a method includes activating a first modality user input mode in which user inputs by way of a first modality are recognized using a first modality recognizer; and receiving a user input by way of the first modality. The method includes, obtaining, as a result of the first modality recognizer recognizing the user input, a transcription that includes a particular term; and generating an input context data structure that references at least the particular term. The method further includes, transmitting, by the first modality recognizer, the input context data structure to a second modality recognizer for use in updating a second modality recognition model associated with the second modality recognizer.