Actionable Visual Search Entity Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current search technologies are inadequate for providing actionable information about entities identified in visual queries, particularly in real-time, without requiring users to input specific terms, such as finding information about a location or entity directly visible to the user.
Innovation Solution
A system that processes visual queries by identifying entities in images and generating actionable search results, allowing users to perform actions like calling, messaging, or mapping directly from the visual query, using a server system that sends actionable search result elements to the client device, which can include buttons for initiating client-side actions such as phone calls, emails, or mapping.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If text-based searching is used, then users can find information about entities, but users must input relevant terms manually which reduces ease of operation
Solution Approach 1:
The system automatically captures the entity from the visual query image and performs the search without requiring the user to manually input terms. The client device takes a picture of the entity, the server identifies the entity in the image, and automatically searches for and returns information about that entity, making the system serve itself rather than requiring manual user input for each search term.
2Ease of operation
If visual query processing is implemented, then ease of operation is improved, but the system complexity increases due to image processing requirements
Solution Approach 1:
The patent introduces an intermediary entity recognition system that bridges the visual query and information retrieval. The server includes an entity recognition component that processes the image to identify the target entity, then uses this identified entity to query information databases. This intermediary processing layer handles the complexity of image analysis while presenting a simple visual interface to users.
3Productivity
If actionable search result elements are generated, then user productivity is improved through direct actions, but the search result processing complexity increases
Solution Approach 1:
The system performs preliminary actions by pre-processing the search results to identify actionable information and generating appropriate action buttons before presenting results to the user. When an entity is identified in the visual query, the system proactively searches for associated actions (calling, messaging, mapping) and prepares actionable search result elements in advance, so that when results are displayed, users can immediately execute actions without additional processing steps.
Data Source
AI summary
A server system receives a visual query and information about the current location of the querying device. The server system sends the query and location information to a visual query search system and receives one or more search results in accordance with the visual query and the location information. From the one or more search results, the server system identifies an entity in the visual query. The server system further identifies a client-side action corresponding to the identified entity and creates an actionable search result element configured to launch the client-side action. For example, if the identified entity were a restaurant the actionable search result elements could include buttons to initiate a telephone call, to read reviews, to map the address, or to make a reservation. If the actionable search result is selected, the client system launches the client-side action in an application distinct from the visual query client application.