Contextual Image Analysis Service for Entity-Based Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current search engines face difficulties in returning contextually relevant results when an image is used as input, as they often rely on visual similarity and lack integration of contextual data, limiting their ability to provide meaningful and relevant information to users.
Innovation Solution
A contextual image analysis service that generates and surfaces contextually relevant data objects by identifying entities and keywords from image content, transforming them into queries, and integrating relevant data objects back into the image context, such as trivia, factual data, or advertisements, to enhance search results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If search engines use visual similarity to return image results, then the search process is simple and fast, but the results lack contextual relevance and meaningful information
Solution Approach 1:
The patent segments the image into multiple regions and identifies different entities within each region (e.g., objects, scenes, actions). This allows the system to generate multiple targeted queries for different entities rather than treating the image as a single unit, thereby improving contextual relevance while managing processing complexity through divided analysis
Solution Approach 2:
The patent introduces an intermediary layer of entity recognition and query generation between the image input and search results. This intermediary process converts visual content into structured entity labels and contextual queries, which then guide the search engine to return more relevant results without requiring the entire system to become exponentially more complex
2Loss of information
If image understanding processing generates keywords to describe image content, then the processing is straightforward, but the results lack context and do not provide meaningful information for search
Solution Approach 1:
The patent transforms the parameter of image description from simple keywords to structured entity labels with contextual attributes. By changing the parameter representation from basic object names to enriched entity structures containing relationships and contextual properties, the system retains more information while maintaining search service efficiency through structured data formats
3Adaptability or versatility
If search engines return only visually similar images, then the processing is efficient and fast, but the user experience is limited and does not provide diverse contextual information
Solution Approach 1:
The patent performs preliminary entity recognition and query generation based on the uploaded image before the actual search is executed. By pre-processing the image to identify entities and generate contextual queries in advance, the system prepares multiple search directions upfront, enabling faster retrieval of diverse results without requiring extensive real-time processing when the user views results
Data Source
AI summary
Non-limiting examples of the present disclosure describe processing by a contextual image analysis service that generates and surfaces contextually relevant data objects based on entities identified from image content. In one example, image content is analyzed and entity annotations are generated for the image content. The entity annotations may be converted to queries. Raw search results may be identified based on the converted queries. The raw search results may be filtered based on one or more specific content types. A contextual representation is generated based on the filtered raw search results, where the contextual representation comprises the image content and one or more contextually relevant data objects for at least one entity associated with the image content. The exemplary contextual representation of the image content and the one or more contextually relevant data objects may be surfaced through a user interface of an application/service.