Voice-Guided Screen Object Control Using Pre-Mapped Text-Image Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing electronic apparatuses face challenges in accurately and efficiently controlling applications, particularly those provided by third parties, due to limitations in user input methods and delays in screen capture and text recognition, especially in diverse form factors like HMD devices.
Innovation Solution
An electronic apparatus equipped with a microphone, memory, and processor that processes user voice to identify corresponding screen images and control objects based on pre-mapped text-image information, allowing for improved accuracy and reduced delay by using high-resolution image processing and voice commands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If real-time screen capture and text recognition are performed, then control responsiveness is improved, but processing time and computational resources are increased
Solution Approach 1:
The patent pre-extracts and stores text information and image data from the screen before voice commands are received. This preliminary action allows the system to quickly match voice commands with pre-processed screen elements, eliminating the need for real-time text recognition and significantly reducing processing delay while maintaining responsiveness.
Solution Approach 2:
The patent divides the screen into multiple image elements and extracts text information from each segment independently. This segmentation allows parallel processing of different screen regions, reducing the overall processing time while maintaining accurate text recognition and control responsiveness.
2Measurement precision
If text recognition accuracy is improved through high-resolution processing, then control precision is improved, but processing speed is reduced
Solution Approach 1:
The patent performs high-resolution text extraction and image processing in advance, storing the results for quick retrieval. This preliminary high-quality processing eliminates the need to choose between resolution and speed during actual voice commands, as the accurate data is already prepared and accessible instantly.
Solution Approach 2:
The patent creates copies of screen images and extracted text data that can be quickly accessed and matched against voice commands. These copies allow the system to maintain high processing speed while referring to pre-processed high-resolution data, effectively decoupling the speed-quality trade-off.
3Ease of operation
If voice command processing is added to existing control methods, then ease of operation is improved, but device complexity is increased
Solution Approach 1:
The patent integrates voice command processing with existing screen capture and text recognition systems, making the system multi-functional. The same infrastructure handles both traditional control methods and voice-based control, reducing the need for separate dedicated systems and thereby limiting the increase in device complexity while improving ease of operation.
Solution Approach 2:
The patent introduces a voice recognition module as an intermediary layer that converts voice commands into text, which then interacts with the existing screen element identification system. This intermediary approach allows voice control to be added without fundamentally redesigning the core control architecture, thus managing complexity while enhancing user convenience.
Data Source
AI summary
Provided is an electronic apparatus including: a microphone; memory storing instructions; and a processor configured to execute the instructions, wherein the instructions, when executed by the processor, cause the electronic apparatus to: based on receiving a user voice through the microphone while a screen including a plurality of first images is being output, obtain text corresponding to the user voice; identify a second image corresponding to the obtained text from among information about at least one text and an image corresponding to the at least one text, wherein the information about the at least one text and the image corresponding to the at least one text are stored in the memory; and based on the user voice and a captured image of the screen, control an object in an area of the screen corresponding to the identified second image.