Contextual Information Retrieval for Image Resources
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for providing contextual information to users about displayed resources require users to input queries, leading to inefficiencies in processing and bandwidth usage, and may not function effectively when users cannot type or are in situations where input is limited.
Innovation Solution
A system and method where a device provides contextual information to a user without requiring a query input, by initially transmitting textual content of a displayed resource to a server, and only transmitting images if textual content is insufficient to provide contextual information, thereby reducing bandwidth and power usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If users input queries to obtain contextual information, then users can obtain contextual information about displayed resources, but processing resources and bandwidth are wasted due to input errors and erroneously input queries
Solution Approach 1:
The system automatically extracts contextual information from the displayed resource itself without requiring user queries. The device detects the active resource, extracts its content, and uses this content to retrieve contextual information, making the system self-sufficient and eliminating user input errors
Solution Approach 2:
The system copies the content from the displayed resource and uses this copied content as the basis for retrieving contextual information, rather than relying on user-generated queries. This ensures the retrieved information is always relevant to what the user is actually viewing
2Reliability
If users save and upload images as queries, then users can obtain contextual information for image-based resources, but bandwidth usage and power consumption increase significantly
Solution Approach 1:
The system segments the approach by first attempting to use textual content from the displayed resource for contextual retrieval, and only transmitting image content if textual content is insufficient. This hierarchical segmentation reduces bandwidth usage by avoiding unnecessary image transmissions
Solution Approach 2:
The system performs preliminary extraction of textual content from the displayed resource before attempting to retrieve contextual information. This preliminary action allows the system to use lighter-weight text-based retrieval methods first, reserving heavier image-based methods only when absolutely necessary
3Reliability
If users manually input queries, then users can search for contextual information, but the process becomes inconvenient when users have limited mobility or cannot type
Solution Approach 1:
The system automatically detects the user's active resource and initiates contextual information retrieval without requiring any user input. The device monitors the active resource, extracts its content, and automatically performs the retrieval process, making the system accessible to users with limited mobility or typing ability
Solution Approach 2:
The system replaces the mechanical action of manual query input with automated electronic detection and extraction processes. The device uses sensors and software to automatically identify and extract content from the displayed resource, substituting physical user actions with electronic automation
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
Systems, methods and software for providing contextual information for a displayed resource that includes an image. A user device provides, to a contextual information server, a query-independent request for contextual information relevant to a resource displayed in an application environment on the user device. The request specifies content of the active resource and further specifies that the active resource displayed on the user device includes an image, but does not include the image in the request. The user devices receives a request for the image from the contextual information server and provides the image to the contextual information server, and receives a user interface element that includes contextual information regarding the image, and displays the user interface element on the user device with the active resource. The methods and systems reduce processing and bandwidth requirements and/or provide other technical advantages compared to known methods and systems of providing contextual information.