Screen-Context Voice Recognition for New On-Screen Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing voice recognition systems struggle to accurately interpret user utterances related to new content or functions on electronic device screens due to insufficient training, leading to poor performance and resource constraints.
Innovation Solution
An electronic device generates a data structure based on screen objects, using image processing and metadata, to interpret voice commands by identifying object types and priorities, enabling accurate recognition of user intents and performing corresponding operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If voice recognition is trained on general data, then the system can recognize common commands, but it fails to accurately recognize new content or functions on screen
Solution Approach 1:
The system performs preliminary action by extracting text information from the screen display before voice recognition occurs. The screen content is processed to identify objects and their attributes in advance, creating a contextual framework that enables accurate recognition of new content without requiring retraining. This preliminary extraction of visual information establishes the basis for subsequent voice command interpretation.
Solution Approach 2:
The patent introduces an intermediary mechanism that combines screen text extraction with voice recognition. The extracted screen information acts as a mediator between the voice command and the intended function, enabling the system to interpret voice commands in the context of currently displayed content. This intermediary layer allows the system to bridge general voice recognition with specific screen context, resolving the contradiction between accuracy and adaptability.
2Measurement precision
If the system processes screen content and voice commands, then recognition accuracy improves, but processing time and computational resources increase
Solution Approach 1:
The system applies segmentation by dividing the processing task into distinct modules: screen content extraction, object identification, and voice command interpretation. Each module processes information independently and passes results to the next stage, enabling efficient processing. The screen text is extracted separately from voice commands, and objects are identified independently, reducing the computational burden of processing everything simultaneously while maintaining high accuracy.
3Measurement precision
If the system identifies object types and priorities from screen content, then voice command interpretation improves, but device complexity increases
Solution Approach 1:
The patent implements universality by creating a multi-functional system where the same screen extraction and object identification mechanisms serve multiple purposes: they provide context for voice recognition, enable accurate interpretation of new content, and reduce the need for separate specialized components. The object identification module serves both as a contextual analyzer and as a disambiguation tool for voice commands, reducing overall system complexity while maintaining high interpretation accuracy.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enhances voice recognition accuracy by mapping user commands to screen functions, improving performance on new content and applications without requiring extensive retraining.
Implementation Method 1
identifying the one or more objects displayed on the screen based on image information or text information, the image information being displayed on an object layer of the screen, the text information being displayed on a text layer of the screen
Data Source
AI summary
A method and an electronic device for providing a voice recognition service are provided. The method includes: receiving a voice command of a user while one or more objects are displayed on a screen of the electronic device; based on the receiving the voice command, identifying the one or more objects displayed on the screen; interpreting text converted from the voice command, based on types of the one or more objects; and performing an operation related to an object selected from among the one or more objects, based on a result of interpreting the text, wherein the types of the one or more objects are identified based on whether the one or more objects are selectable by a user input to the electronic device.