Visual Search Overlays for In-App Image Object Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image viewing applications on computing devices require users to switch between applications for object identification and information retrieval, hampering the viewing experience by adding time and effort.
Innovation Solution
A system that generates visual indicators on images or videos displayed by any application, allowing users to select and obtain information about elements of interest directly without navigating away, using machine learning models to identify and rank elements of interest and provide selectable visual indicators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If users switch between applications for object identification and information retrieval, then information can be obtained, but viewing experience is hampered by additional time and effort
Solution Approach 1:
The patent combines object identification and information retrieval functions into a single application by integrating a machine learning model that detects elements of interest and generates visual indicators directly on the displayed image, allowing users to obtain information without switching applications
Solution Approach 2:
The patent introduces visual indicators as an intermediary element that bridges the gap between image viewing and information retrieval, providing direct access to information about detected elements through selectable indicators overlaid on the image
2Loss of information
If visual indicators are generated for all elements in an image, then complete information is provided, but device complexity and processing load increase
Solution Approach 1:
The patent applies local quality by generating visual indicators selectively for specific elements of interest detected in the image rather than for all elements, allowing the system to focus processing on relevant objects and reduce overall complexity
Solution Approach 2:
The machine learning model automatically identifies and ranks elements of interest, enabling the system to self-determine which elements require visual indicators without manual intervention, reducing the burden on users and optimizing processing
Data Source
AI summary
The subject technology provides visual search systems and methods that can be used to efficiently perform visual searches on an electronic device. The subject technology provides systems and methods for presenting one or more visual indicators corresponding to the searchable portions of digital content. A visual search may include identifying, at an electronic device, an element of interest in an image. A visual indicator for the element of interest may be overlaid on the image at a location corresponding to the element of interest. The visual indicator may be selectable to cause a display of information associated with the element of interest responsive to the selection.