Voice Recognition Using Screen Context Language Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current voice recognition systems for electronic devices face challenges in improving performance, maintaining accuracy with inconsistent user inputs, and reducing memory usage while providing a consistent user experience, especially when handling inaccurate data and special characters.
Innovation Solution
The system extracts context information from the device's screen to generate a real-time language model, which is used to convert voice inputs into text strings and reset the model dynamically, enhancing voice recognition performance and accuracy while reducing memory usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a conventional voice recognition system is used, then the system structure is simple, but the voice recognition accuracy is insufficient
Solution Approach 1:
The patent introduces a language model as an intermediary component between the voice input and the recognition result. This language model uses context information from the screen to assist the voice recognition process, thereby improving accuracy without requiring complete system redesign. The language model acts as a mediator that bridges the gap between raw voice input and accurate interpretation.
Solution Approach 2:
The system makes the display screen serve multiple functions: it not only displays information to the user but also provides context information for voice recognition. By extracting text and contextual data from the screen content, the system leverages an existing component (the display) to enhance voice recognition accuracy, reducing the need for additional dedicated hardware.
2Measurement precision
If a large language model is used to improve recognition accuracy, then the voice recognition performance improves, but the memory usage increases
Solution Approach 1:
Instead of using a comprehensive large language model that processes all possible contexts, the system extracts only the specific context information currently displayed on the screen. This localized approach focuses computational resources on the immediately relevant context, improving recognition accuracy for the current task while minimizing memory consumption by avoiding unnecessary model components.
Solution Approach 2:
The system uses a partial language model that processes only the context information from the screen rather than a complete language model that would process all possible linguistic contexts. This partial action approach provides sufficient recognition accuracy for the current display context while using significantly less memory than a full language model would require.
3Adaptability or versatility
If the language model continuously learns from all inputs, then the recognition adapts to user patterns, but the processing time and memory consumption increase
Solution Approach 1:
The system resets the language model periodically rather than continuously learning from all inputs. By clearing and regenerating the language model based on current screen context, the system maintains adaptability to the current display state while avoiding the cumulative processing overhead of continuous learning from all historical inputs.
Solution Approach 2:
The system extracts and prepares context information from the screen in advance before voice recognition occurs. By having the context information ready beforehand, the language model can be quickly generated or reset without requiring real-time processing during voice input, thereby reducing overall processing time while maintaining adaptability.
Data Source
AI summary
An electronic device and method are disclosed. The electronic device includes input circuitry, a display, and a processor. The processor implements the method, including extracting at least one piece of context information based at least in part on an application screen displayed on the display, analyzing the extracted at least one piece of context information to generate a language model based on the extracted at least one piece of context information, receiving a voice input of a user through the input circuitry and convert the voice input into a text string using the generated language model, and resetting the generated language model.