Invisible Light Markers for Augmented Reality Overlay

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

In augmented reality systems, visible markers used for overlaying virtual objects on real-world images can be obstructive, easily copied, and require complex detection methods, which limits their application and user experience.

Innovation Solution

A composite image generating system that uses invisible light markers, where a camera captures both visible and invisible light images, and an image processing apparatus determines the overlaying conditions to position virtual objects without obstructing the real-world view and prevents marker copying by hiding or erasing the marker image.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If visible markers are used for tracking and overlaying virtual objects, then the overlaying position and orientation can be accurately determined, but the markers obstruct the real-world view and can be easily copied

Engineering Contradiction:
Improveoverlaying position determination accuracyVSAvoidmarker obstruction and copyability
Core Design Contradiction:
Measurement precisionVSObject-generated harmful factors

Solution Approach 1:

The patent applies the principle of color changes by transitioning from visible markers to invisible markers that reflect or radiate infrared light. The markers become undetectable to the human eye while remaining detectable by infrared cameras, thus eliminating the obstruction problem while maintaining tracking functionality. This is achieved by changing the optical properties of the markers from visible spectrum to infrared spectrum.

Inventive Principle:
Principle #32Color changes

Solution Approach 2:

The patent introduces an intermediary element - the infrared camera - as a mediator between the marker and the detection system. Instead of directly detecting visible markers with the camera, the system uses infrared light as an intermediary carrier that reflects off the markers and is captured by the infrared camera, enabling indirect detection without visible obstruction.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Object-generated harmful factors

If invisible light markers are used, then the real-world view is not obstructed and copying is prevented, but the detection and discrimination of marker conditions becomes more complex

Engineering Contradiction:
Improvemarker obstruction and copyabilityVSAvoidmarker condition discrimination complexity
Core Design Contradiction:
Object-generated harmful factorsVSDifficulty of detecting and measuring

Solution Approach 1:

The patent applies inversion by reversing the detection approach - instead of the camera actively emitting light and detecting reflections (active detection), the system uses passive detection where the marker itself radiates or reflects infrared light that is then captured. This inversion simplifies the detection mechanism while maintaining accuracy.

Inventive Principle:
Principle #13The other way round (Inversion)

Solution Approach 2:

The invisible markers are designed to self-identify through their unique infrared reflection or radiation patterns. The markers contain embedded identification information that automatically serves the detection system, eliminating the need for complex external verification systems and reducing overall system complexity.

Inventive Principle:
Principle #25Self-service

3Ease of manufacture

If standard camera technology is used to capture invisible light images, then cost is reduced, but the ability to discriminate both visible and invisible light images simultaneously is limited

Engineering Contradiction:
Improvesystem costVSAvoiddual light spectrum capture capability
Core Design Contradiction:
Ease of manufactureVSAdaptability or versatility

Solution Approach 1:

The patent applies multi-functionality by designing the infrared camera to serve dual purposes: capturing invisible infrared markers for AR tracking and capturing visible light images for display to the user. This single device performs multiple functions that would otherwise require separate systems, reducing overall system cost and complexity.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent combines the infrared detection function and visible light imaging function into a single camera system. By merging these two previously separate functions into one device, the system achieves cost-effectiveness while maintaining the ability to simultaneously process both invisible marker data and visible real-world imagery.

Inventive Principle:
Principle #5Merging (Combining)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This solution prevents visible markers from obstructing the real-world view, makes them difficult to copy, and allows for cost-effective implementation by using standard camera technology, enhancing user experience and system efficiency in augmented reality applications.

Implementation Method 1

at least one mark which at least reflects or radiates invisible light of predetermined wavelength other than visible light

Methodology Applied
Scientific EffectInfrared reflection: Reflection

Implementation Method 2

at least one mark which at least reflects or radiates invisible light of predetermined wavelength other than visible light

Methodology Applied
Scientific EffectInfrared radiation: Infrared Radiation

Data Source

PatentEP2279500B1Composite image generating system, overlaying condition determining method, image processing apparatus, and image processing program
Publication Date: 2018.10.31 APPLE INC
  • EP2279500B1 patent drawingFigure 1
  • EP2279500B1 patent drawingFigure 2
  • EP2279500B1 patent drawingFigure 3

AI summary

A mark which at least reflects or radiates invisible light of predetermined wavelength other than visible light is added in a space of the real world. A camera apparatus comprises image capturing means for capturing a real image in which an invisible-light image may be discriminating. An image processing apparatus, comprises: mark discriminating means for discriminating at least one condition of a position of image of mark in the captured real image, a orientation of the mark, and a distance from the mark to the image capturing means, and overlaying condition determining means for determining, in correspondence with the discriminated condition, a overlaying condition which is at least one of an overlaying position which is a position of the image of the virtual object overlaid on the captured real image in the real image, an orientation of the virtual object which the image of the virtual object indicates, and a distance from a view point of viewer, of the image of the virtual object.