AR Camera Calibration Using Dual-View Marker Detection

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing head-mounted display (HMD) systems face challenges in accurately estimating the position and posture of the display section due to changes in camera parameters over time or usage environment, requiring complex calibration setups that are not practical for user-level adjustments.

Innovation Solution

An information processing apparatus that acquires first and second estimation information from captured images using a camera worn by the user and a second camera in the user's environment, respectively, to generate correction information for calibrating camera parameters, allowing for accurate estimation of the display section's position and posture without the need for observation cameras at specific user-eye positions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If observation cameras are arranged at positions corresponding to user eyes to estimate positional relation between display section and camera, then estimation accuracy is improved, but device complexity and installation difficulty increase

Engineering Contradiction:
Improveestimation accuracy of positional relationVSAvoidcalibration setup complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent uses a marker (copy object) placed in the real world that can be captured by both the first camera (worn by user) and second camera (fixed in environment). By detecting the marker's position and posture from both cameras, the system establishes correspondence between the two cameras without requiring observation cameras at user eye positions. This copying approach simplifies the calibration setup while maintaining estimation accuracy.

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The patent introduces a marker as an intermediary object that mediates the calibration process between the first camera and second camera. The marker serves as a common reference point that both cameras can observe, enabling the calculation of positional relations without direct observation of user eye positions. This intermediary approach resolves the contradiction by providing a practical calibration method that achieves accuracy without complex setup.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Measurement precision

If dedicated calibration environment with observation cameras is used, then calibration accuracy is improved, but ease of operation deteriorates due to specialized requirements

Engineering Contradiction:
Improvecalibration accuracyVSAvoidcalibration accessibility
Core Design Contradiction:
Measurement precisionVSEase of operation

Solution Approach 1:

The patent enables the calibration system to perform self-calibration by using the marker detected from images captured by the first and second cameras. The calibration processing section automatically calculates correction information based on the detected marker positions and postures, without requiring manual intervention or specialized calibration environments. This self-service approach allows users to perform calibration in ordinary environments, significantly improving ease of operation while maintaining accuracy.

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The patent makes the calibration process universal by using a simple marker that can be placed in any ordinary environment rather than requiring dedicated calibration facilities. The same calibration approach can be applied regardless of the specific environment, making the system adaptable and easy to operate in various settings. This multi-functionality resolves the contradiction between accuracy and accessibility.

Inventive Principle:
Principle #6Universality (Multi-functionality)

3Manufacturing precision

If camera parameters are calibrated at factory shipment, then initial accuracy is improved, but reliability deteriorates over time due to parameter changes from aging and environment

Engineering Contradiction:
Improveinitial parameter accuracyVSAvoidparameter stability over time
Core Design Contradiction:
Manufacturing precisionVSReliability

Solution Approach 1:

The patent performs calibration at factory shipment as a preliminary action to establish initial accurate parameters. The calibration processing section calculates correction information based on marker detection, and this correction information is stored and applied during actual use. This preliminary calibration ensures both initial accuracy and long-term reliability by providing a baseline that compensates for future parameter drifts.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent implements a feedback mechanism where the system periodically performs calibration by detecting the marker and recalculating correction information. This feedback loop allows the system to adapt to parameter changes caused by aging and environmental factors, maintaining reliability over time. The calibration results from previous operations serve as feedback to update and refine the correction information, ensuring long-term parameter stability.

Inventive Principle:
Principle #23Feedback

Data Source

PatentUS20240169590A1Information processing apparatus, information processing method, and storage medium
Publication Date: 2024.05.23 SONY GROUP CORP
  • US20240169590A1 patent drawing
  • US20240169590A1 patent drawing
  • US20240169590A1 patent drawing

AI summary

An information processing apparatus includes an acquiring section that acquires first estimation information representing a position and a posture of a display section estimated on the basis of a first captured image captured with a first camera worn by a user along with the display section, and second estimation information representing a position and a posture of the display section estimated on the basis of a second captured image captured with a second camera installed in a space where the user exists, and a calibration processing section that generates correction information used for calibration of a parameter of either the first camera or the second camera on the basis of the first estimation information and the second estimation information.