Screen Object Search via Image Capture and AI Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users face inconvenience in retrieving information related to objects on an electronic device screen, as they need to perform multiple steps and often re-search due to lack of image search functionality, especially when the device does not support image searches, requiring manual text input.
Innovation Solution
A method and device that allow users to select an object on the screen, generate an image, and use trained models to recognize areas of the image for providing a search result, combining first and second information from different areas, with an AI agent performing these operations and transmitting feedback for improved search results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual text input is used for searching object information, then search functionality is available, but user convenience deteriorates and operation time increases
Solution Approach 1:
The patent replaces manual text input (mechanical typing action) with image-based search technology. The system captures an image of the object, processes it through AI algorithms to extract visual features, and performs search based on these features automatically, eliminating the need for manual text entry and significantly improving search convenience while reducing time consumption.
Solution Approach 2:
The system enables self-service search by automatically capturing images, processing them through AI models, and retrieving search results without requiring user intervention for text input. The electronic device performs the entire search process autonomously once the user selects the image search function, allowing the system to serve itself in completing the search task.
2Loss of information
If multiple search steps are performed to retrieve comprehensive information, then information completeness improves, but operation complexity increases
Solution Approach 1:
The patent merges multiple search operations into a single integrated image search process. Instead of requiring separate searches for different aspects of an object, the system processes the object image once and retrieves comprehensive information including basic attributes, detailed specifications, and related content in a unified search result, thereby maintaining information completeness while reducing operational complexity.
Solution Approach 2:
The image search system performs multiple functions simultaneously: it identifies the object, extracts visual features, searches for matching information, and presents comprehensive results covering various aspects of the object. This multi-functional approach replaces the need for multiple specialized search operations, simplifying the user interface while maintaining thorough information retrieval.
3Measurement precision
If image search functionality is implemented, then search accuracy improves, but device complexity increases
Solution Approach 1:
The patent introduces an intermediary AI processing system that bridges the gap between simple image capture and complex search requirements. The AI model acts as a mediator that automatically processes the captured image, extracts relevant visual features, and translates them into effective search queries, thereby achieving high search accuracy without requiring complex user-side image processing capabilities.
Solution Approach 2:
The system performs preliminary image processing and feature extraction automatically before the actual search operation. By pre-processing the image through AI algorithms to extract key visual characteristics and object attributes, the system prepares optimized search queries in advance, achieving high search accuracy while keeping the user-facing interface simple and the apparent device complexity low.
Data Source
AI summary
A method of providing a search result in an electronic device and an electronic device therefor, are provided. The method includes displaying a screen including an object and sensing a user input of selecting the object. In this method, in response to the user input, the screen including the object is captured to generate the captured image. The method further includes transmitting at least part of the captured image to an external device, receiving a search result, which is obtained based on object information corresponding to the object and additional information related to the object from an external device, and displaying the received search result.