HMD Exposure Timing Synchronization for Mixed Reality

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Mixed reality (MR) systems using head-mounted displays (HMDs) face issues with insufficient calculation accuracy due to asynchronous operations between imaging units and sensors, leading to positional shifts and discomfort caused by variations in delay time and frame rates.

Innovation Solution

A head-mounted display apparatus that includes a display unit, a first imaging unit using a rolling shutter method, a second imaging unit using a global shutter method, and a generation unit that generates synchronization signals to control the exposure start times of both imaging units based on image input timing, ensuring synchronous operations with an orientation sensor.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If common driving signals are supplied to multiple imaging units to synchronize them, then synchronization between imaging units is improved, but synchronization with sensors and display unit remains insufficient

Engineering Contradiction:

Improvesynchronization accuracyVSAvoidsystem coordination capability

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent segments the synchronization control into separate pathways: one for imaging units and another for sensors and display units. The generation unit independently generates driving signals for imaging units based on image input timing signals from the display unit, while separately controlling sensors. This segmentation allows each component to be synchronized independently to the display timing, resolving the contradiction between imaging unit synchronization and overall system coordination.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The generation unit acts as an intermediary that receives the image input timing signal from the display unit and translates it into coordinated driving signals for both imaging units and sensors. This intermediary function enables centralized timing coordination across all components based on the display unit's timing requirements, ensuring that imaging units, sensors, and display unit all operate in synchronization.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Device complexity

If imaging units operate asynchronously with sensors, then device complexity is reduced, but calculation accuracy deteriorates due to positional shifts

Engineering Contradiction:

Improvesynchronization control complexityVSAvoidposition calculation accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The system performs preliminary synchronization by having the generation unit generate driving signals for imaging units and sensors in advance based on the image input timing signal from the display unit. This ensures that all components are pre-coordinated to capture data at the correct timing intervals, maintaining calculation accuracy without requiring complex real-time synchronization adjustments.

Inventive Principle:

Principle #10Preliminary action

3Device complexity

If exposure timings of multiple imaging units are not aligned, then device complexity is reduced, but image quality deteriorates due to temporal shifts

Engineering Contradiction:

Improveexposure control complexityVSAvoidimage capture precision

Core Design Contradiction:

Device complexityVSManufacturing precision

Solution Approach 1:

The patent applies equipotentiality by aligning the exposure timings of multiple imaging units to a common reference timing derived from the image input timing signal. This creates a synchronized timing baseline for all imaging units, ensuring they capture images at equivalent temporal points, thereby eliminating temporal shifts and improving image capture precision without requiring complex individual timing control for each unit.

Inventive Principle:

Principle #12Equipotentiality

4Adaptability or versatility

If frame rates of imaging units and display unit are not coordinated, then adaptability is improved, but user comfort deteriorates due to delay variations and missing frames

Engineering Contradiction:

Improveframe rate flexibilityVSAvoiduser comfort

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The system implements feedback by using the image input timing signal from the display unit as a reference to generate driving signals for imaging units. This feedback mechanism ensures that the frame rates of imaging units are coordinated with the display unit's refresh rate, maintaining smooth image presentation and preventing delay variations or missing frames that would cause user discomfort.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS20240185543A1Head-mounted display apparatus that controls exposure timings of plurality of imaging units

Publication Date: 2024.06.06 CANON KK

AI summary

A head-mounted display apparatus includes a display unit, a first imaging unit, a second imaging unit, and a generation unit configured to, based on a signal indicating an image input timing of input of an image to the display unit, generate a first signal for controlling a start of exposure of the first imaging unit and a second signal for controlling a start of exposure of the second imaging unit.