Image Entity Recognition via Similar Image Metadata Analysis

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current search engines face challenges in accurately determining the subject matter of an image query, as users often receive a set of related images but must manually identify the subject, which can be time-consuming and inefficient.

Innovation Solution

A system and method where a search engine identifies similar images to a query image, extracts metadata, and uses machine learning to determine a most-likely entity name, matching it to a known entity to generate relevant search results pages.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If a search engine provides only similar images in response to an image query, then the user can view related images, but the user must manually identify the subject matter which is time-consuming and inefficient

Engineering Contradiction:
Improvesubject matter identificationVSAvoidmanual sleuthing time
Core Design Contradiction:
Loss of informationVSLoss of time

Solution Approach 1:

The system performs preliminary analysis of image metadata and determines the subject matter before presenting results to the user. Search result pages are pre-generated with identified entity information, eliminating the need for users to manually investigate each image to determine its subject matter.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent introduces an intermediary processing layer that analyzes image metadata and generates search result pages with identified subject matter. This intermediary system acts as a mediator between the image database and the user, automatically determining entity names and presenting organized information without requiring direct user investigation of individual images.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Measurement precision

If the search engine analyzes metadata from multiple similar images to determine entity names, then accurate subject matter identification is achieved, but the processing complexity and time increase

Engineering Contradiction:
Improveentity name accuracyVSAvoidmetadata analysis complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The system analyzes metadata from a selected subset of similar images rather than all available images. By processing only the necessary portion of metadata required to determine the subject matter with sufficient accuracy, the system achieves effective entity identification without the excessive complexity of analyzing every possible metadata element from every similar image.

Inventive Principle:
Principle #16Partial or excessive action

3Loss of information

If the search engine generates comprehensive search result pages with entity information, then users receive complete information about the subject matter, but the time and resources required to generate these pages increase

Engineering Contradiction:
Improveentity information completenessVSAvoidsearch result generation speed
Core Design Contradiction:
Loss of informationVSProductivity

Solution Approach 1:

The system pre-generates search result pages with entity information before user requests. By performing the information gathering and organization in advance, the system can quickly retrieve and present comprehensive entity details when a user submits an image query, without having to generate the information on-demand.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentEP3289489B1Image entity recognition and response
Publication Date: 2020.11.25 MICROSOFT TECHNOLOGY LICENSING LLC
  • EP3289489B1 patent drawingFigure 1
  • EP3289489B1 patent drawingFigure 2
  • EP3289489B1 patent drawingFigure 3

AI summary

Systems and methods for responding to an image query from a computer user are provided. According to the disclosed subject matter, in response to receiving an image query, a search engine identifies the subject matter of the query image according to similar images. An entity name is determined from the similar images and is mapped to a known entity of the search engine. Based on the known entity, related information regarding the known entity is obtained and one or more search results pages directed to the known entity are generated. At least one of the generated search results pages is returned to the computer user as a response to the image query.