Automated Landmark Detection Using Appearance Models

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional digital image organizing systems rely on manual user tagging, which is inefficient and inconsistent, especially in large collections of images, making it difficult to automatically identify and annotate popular landmarks.

Innovation Solution

A system and method for automatically detecting and annotating landmarks in digital images by learning an appearance model from tagged images, using a combination of text-associated data, geo-location information, and visual features to accurately identify and classify popular landmarks.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual user tagging is used to organize digital images, then users can provide descriptive labels, but the process becomes inefficient and inconsistent when dealing with large collections of images

Engineering Contradiction:
Improvetagging consistencyVSAvoidtagging efficiency
Core Design Contradiction:
Measurement precisionVSProductivity

Solution Approach 1:

The system enables automatic self-tagging of images by extracting landmark information without requiring manual user input. The automated landmark detection system processes images independently, generating tags based on visual analysis of landmark features, thereby eliminating the need for manual tagging while maintaining consistency and efficiency.

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The patent replaces the manual mechanical process of user tagging with an automated computational system. The system uses image processing algorithms, visual feature extraction, and machine learning models to automatically identify and tag landmarks, substituting human labor with automated mechanical processes that operate continuously without fatigue or inconsistency.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

2Productivity

If automated landmark detection is implemented, then processing speed increases, but the system complexity increases

Engineering Contradiction:
Improveimage processing speedVSAvoidsystem complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The automated detection system is divided into distinct functional modules: image input module, landmark detection module, feature extraction module, tag generation module, and annotation module. Each segment performs a specific function independently, making the complex system manageable and easier to implement while maintaining high processing speed through parallel operation of these segments.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The system introduces intermediate representation layers between image input and final tag generation. Visual features serve as intermediaries that bridge the gap between raw images and text tags, while a knowledge base acts as an intermediary that stores landmark information and relationships, enabling efficient automated detection without excessive system complexity.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Measurement precision

If appearance models are learned from tagged images, then landmark detection accuracy improves, but the initial data preparation requirement increases

Engineering Contradiction:
Improvelandmark detection accuracyVSAvoiddata preparation time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The system performs preliminary learning of appearance models using a curated initial dataset of tagged images before processing new images. This preliminary action establishes the foundation for accurate detection, and once learned, the model can process new images rapidly without requiring continuous data preparation, thus investing time upfront to eliminate ongoing time consumption.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The system creates a copied and simplified representation of landmark appearances through appearance models that capture essential visual features. Instead of working with the full complexity of original tagged images, the system uses these distilled models as copies that retain the necessary information for accurate detection while significantly reducing data preparation and processing requirements for new images.

Inventive Principle:
Principle #26Copying

Data Source

PatentUS10303975B2Landmarks from digital photo collections
Publication Date: 2019.05.28 GOOGLE LLC
  • US10303975B2 patent drawing
  • US10303975B2 patent drawing
  • US10303975B2 patent drawing

AI summary

Methods and systems for automatic detection of landmarks in digital images and annotation of those images are disclosed. A method for detecting and annotating landmarks in digital images includes the steps of automatically assigning a tag descriptive of a landmark to one or more images in a plurality of text-associated digital images to generate a set of landmark-tagged images, learning an appearance model for the landmark from the set of landmark-tagged images, and detecting the landmark in a new digital image using the appearance model. The method can also include a step of annotating the new image with the tag descriptive of the landmark.