Dynamic Data Model Selection for AR Camera Pose Tracking

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing augmented reality systems face challenges in accurately determining the pose of a camera relative to a real environment and recognizing objects due to unreliable sensor data and varying environmental conditions, leading to inconsistent system reliability.

Innovation Solution

A method that records environmental parameters such as time, weather, and location to dynamically configure tracking and image recognition systems, using a data model adapted to specific conditions to improve the accuracy of camera pose determination and object recognition, employing optical tracking methods and feature descriptors like SURF and SIFT, and prioritizing sensor usage based on environmental factors.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If a single data model is used for object recognition and pose determination, then the system structure remains simple, but the reliability varies greatly with changing environmental conditions

Engineering Contradiction:
Improvesystem reliabilityVSAvoidadaptability to environmental conditions
Core Design Contradiction:
ReliabilityVSAdaptability or versatility

Solution Approach 1:

The system dynamically selects between different data models (first data model for 2D images, second data model for 3D images) based on environmental conditions detected by sensors. This dynamic adaptation allows the system to maintain high reliability across varying environmental conditions while managing complexity through conditional model selection rather than maintaining all models simultaneously.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The system changes the parameter of data model selection based on environmental parameters detected by sensors (lighting conditions, weather, etc.). By monitoring environmental parameters and selecting the appropriate data model accordingly, the system achieves adaptability to different environmental conditions without requiring a completely restructured system architecture.

Inventive Principle:
Principle #35Parameter changes

2Reliability

If multiple data models are maintained for different environmental conditions, then the reliability under varying conditions improves, but the device complexity increases

Engineering Contradiction:
Improveregistration accuracyVSAvoidsystem complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

Rather than maintaining all data models simultaneously, the system dynamically activates only the appropriate data model based on current environmental conditions. This dynamic approach ensures high registration accuracy under varying conditions while avoiding the complexity of managing multiple active models at once.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The system extracts and uses only the necessary data model for the current environmental condition, rather than maintaining all possible models. By taking out only the relevant model (first data model for 2D, second data model for 3D) based on sensor detection, the system achieves high reliability without the overhead of managing complete model sets.

Inventive Principle:
Principle #2Taking out (Extraction)

3Reliability

If environmental parameter detection and dynamic model selection is implemented, then the robustness to changing conditions improves, but the processing time and computational overhead increase

Engineering Contradiction:
ImproverobustnessVSAvoidprocessing time
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary detection of environmental parameters and pre-selects the appropriate data model before actual object recognition and pose determination begin. This preliminary action allows the system to avoid computational overhead during the main processing phase, maintaining robustness while minimizing processing time loss.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The system uses its own sensor data to automatically determine which data model to use, without requiring external input or manual configuration. This self-service approach streamlines the model selection process, reducing processing time while maintaining robustness across environmental conditions.

Inventive Principle:
Principle #25Self-service

Data Source

PatentEP3550516B1Environmental parameter based selection of a data model for recognizing an object of a real environment
Publication Date: 2022.11.23 APPLE INC
  • EP3550516B1 patent drawingFigure 1A~1B
  • EP3550516B1 patent drawingFigure 2
  • EP3550516B1 patent drawingFigure 3

AI summary

A method for determining the pose of a camera (22, 32) relative to a real environment (40) comprises the following steps: recording at least one image (50) of a real environment using a camera (22, 32), wherein the image includes at least a part of a real object (41); performing a tracking procedure that evaluates information regarding correspondences between features associated with the real object (41) and corresponding features of the real object (41) as contained in the image (50) of the real environment in order to infer the pose of the camera (22, 32); determining at least one parameter of an environmental situation; and performing the tracking procedure depending on the at least one parameter. The method can also be applied analogously in a method for detecting an object of a real environment in an image recorded by a camera.