AR Display Continuity During HMD Disconnection

Solution Overview

Problem

In augmented reality display systems, when a head-mounted display (HMD) is disconnected from a host apparatus and then reconnected, there is a loss of continuity in information, requiring recreation of the environment for AR display, as the captured image is stopped from being supplied, disrupting the AR experience.

Innovation Solution

A display system with a host apparatus and HMD, where the host apparatus includes a first inertial sensor and an image processing unit, and the HMD includes a camera and a second inertial sensor, allowing the image processing unit to associate sensor data and maintain target state estimation using first sensor data even when disconnected, enabling seamless reconnection and continuation of AR processing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If the HMD is disconnected from the host apparatus, then the HMD can be removed for portability or safety, but the captured image stops being supplied and AR display continuity is lost

Engineering Contradiction:

ImproveportabilityVSAvoidAR display continuity

Core Design Contradiction:

Ease of operationVSLoss of information

Solution Approach 1:

The system performs preliminary actions by storing captured image data and sensor data in memory before disconnection occurs. The host apparatus and HMD exchange and save relevant information (images, sensor readings, AR object positions) while still connected, so that when disconnection happens, the system can resume AR display using the pre-stored data without losing continuity.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system creates copies of essential data (captured images, sensor data, AR scene information) and stores them in the memory of both the host apparatus and HMD. This copying mechanism ensures that even when the physical connection is broken, the AR display can continue using the copied data, maintaining information continuity across disconnection events.

Inventive Principle:

Principle #26Copying

2Use of energy by moving object

If the captured image is stopped from being supplied when HMD is disconnected, then power consumption is reduced, but the AR environment must be recreated upon reconnection

Engineering Contradiction:

Improvepower consumptionVSAvoidrecreation time

Core Design Contradiction:

Use of energy by moving objectVSLoss of time

Solution Approach 1:

The system performs preliminary actions by storing captured image data and sensor data in memory before disconnection occurs. The host apparatus and HMD exchange and save relevant information (images, sensor readings, AR object positions) while still connected, so that when disconnection happens, the system can resume AR display using the pre-stored data without losing continuity.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system strategically discards the continuous image supply during disconnection to save power, but recovers the AR display state by retrieving previously stored data from memory. Upon reconnection, the system recovers the AR environment using the saved captured images and sensor data, avoiding the need to recreate everything from scratch.

Inventive Principle:

Principle #34Discarding and recovering

3Device complexity

If sensor data association is performed only when connected, then processing complexity is reduced, but AR target tracking accuracy deteriorates during disconnection

Engineering Contradiction:

Improveprocessing complexityVSAvoidtarget tracking accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The system performs preliminary actions by storing captured image data and sensor data in memory before disconnection occurs. The host apparatus and HMD exchange and save relevant information (images, sensor readings, AR object positions) while still connected, so that when disconnection happens, the system can resume AR display using the pre-stored data without losing continuity.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11327705B2Display system, display method, and display program for displaying image on head-mounted display

Publication Date: 2022.05.10 SEIKO EPSON CORP

AI summary

The display system includes a host apparatus and an HMD. The host apparatus includes an image display unit configured to display, an image transmitted from the host apparatus, in which the image processing unit is configured, when detecting that the host apparatus is connected with the HMD, to associate first sensor data from the first inertial sensor with second sensor data received from the second inertial sensor, to recognize a target in the image captured by the camera, to use the second sensor data to perform image processing, and to cause the image display unit to display the image after the image processing in association with the target, when detecting that the host apparatus is not connected with the HMD, to use the first sensor data to maintain an estimation of a state of the target in the outside scene until the host apparatus is reconnected with the HMD.