Voice-Selectable Display Content Using Screen Recognition Labels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing display devices lack the ability to directly select image content within a content provider using voice recognition, limiting user convenience and accuracy in content selection.
Innovation Solution
A display device equipped with a microphone and a controller that utilizes a screen recognition model to detect content areas on a content provider's screen, output labeling icons, and execute content selection through voice commands, employing both Rule-based and deep learning-based screen recognition models to enhance accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If voice recognition is used for content selection, then user convenience is improved, but the ability to directly select content within a content provider screen is limited
Solution Approach 1:
The patent introduces a screen recognition model as an intermediary between the voice recognition system and the content provider screen. This model captures the screen, identifies content areas, and maps voice commands to specific content regions, enabling direct content selection within the content provider interface while maintaining voice recognition convenience
Solution Approach 2:
The patent replaces the traditional mechanical remote control operation (manual cursor navigation) with a voice-based system enhanced by screen recognition. The screen recognition model automatically identifies content boundaries and coordinates, substituting the mechanical interaction process with an automated vision-based system that enables direct content selection
2Measurement precision
If screen recognition is implemented, then content area detection accuracy is improved, but system complexity increases
Solution Approach 1:
The patent segments the screen recognition task into distinct functional modules: screen capture, contour detection, content area identification, and coordinate mapping. This segmentation allows each module to be optimized independently and facilitates integration with existing voice recognition systems without requiring complete system redesign
Solution Approach 2:
The screen recognition model is designed to be universal and adaptable to different content provider screens. By using general image processing techniques and configurable parameters, the system can recognize content areas across various platforms and interfaces without requiring provider-specific customization, thereby managing complexity while maintaining accuracy
Data Source
AI summary
The present disclosure discloses a display device comprising: a communication unit; a display unit; a microphone; and a control unit which acquires screen image data by capturing a content provider screen including at least one piece of content outputted through the display unit upon receiving a wake-up word, inputs the screen image data to a screen recognition model corresponding to the content provider to detect at least one content region, outputs a labeling icon corresponding to each content region by using coordinate information of the content region corresponding to each of the at least one content regions detected by the screen recognition model, and when a user utterance corresponding to any one of the labeling icons is received, executes the content corresponding to the labeling icon.