HMD Image Display Control for Real-World Task Superimposition

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Users wearing head-mounted displays (HMDs) for virtual reality cannot easily perform tasks in the real world, such as writing notes or using smartphones, as they must temporarily remove the HMD to interact with physical objects.

Innovation Solution

An image display control apparatus that detects target objects within a predetermined distance using a camera and superimposes their images onto the virtual space, allowing users to interact with real-world objects while still seeing the virtual environment.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If the user wears the HMD to experience virtual reality, then the user can be immersed in the virtual world, but the user cannot see the real world and may collide with obstacles

Engineering Contradiction:

Improvevirtual reality immersionVSAvoidcollision risk with obstacles

Core Design Contradiction:

Adaptability or versatilityVSObject-affected harmful factors

Solution Approach 1:

A camera is introduced as an intermediary device to capture images of the real world, and these images are displayed on the HMD screen to serve as a mediator between the user and the physical environment, enabling safe navigation without direct visual contact with obstacles

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent creates a visual copy of the real world by capturing images with a camera and displaying them on the HMD screen, allowing the user to perceive the physical environment through this copy rather than direct vision, thus maintaining VR immersion while reducing collision risk

Inventive Principle:

Principle #26Copying

2Adaptability or versatility

If the user wears the HMD to perform virtual tasks, then the user can interact with virtual objects, but the user cannot perform real-world tasks such as writing or using a smartphone

Engineering Contradiction:

Improvevirtual task capabilityVSAvoidreal-world task performance

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The patent merges the virtual reality display with the real-world camera feed by superimposing captured images onto the virtual space in the HMD view, creating a unified display that allows users to perform both virtual and real-world tasks simultaneously without removing the HMD

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent adds a new dimensional layer to the HMD display by overlaying captured real-world images onto the virtual space, transforming the two-dimensional virtual display into a three-dimensional mixed reality interface that accommodates both virtual and physical task requirements

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

3Ease of operation

If the user removes the HMD to perform real-world tasks, then the user can write notes or use a smartphone, but the user must temporarily stop virtual reality experience

Engineering Contradiction:

Improvereal-world task performanceVSAvoidvirtual reality interruption time

Core Design Contradiction:

Ease of operationVSLoss of time

Solution Approach 1:

The patent enables continuous virtual reality experience by displaying captured real-world images on the HMD screen throughout the task performance, eliminating the need to remove the HMD and thus preventing interruption of the virtual reality experience while allowing real-world tasks to be performed

Inventive Principle:

Principle #20Continuity of useful action

4Loss of information

If virtual objects are displayed superimposed on virtual space to indicate obstacles, then the user can understand obstacle positions, but the user still cannot perform hands-on work in the real world

Engineering Contradiction:

Improveobstacle position informationVSAvoidhands-on real-world work capability

Core Design Contradiction:

Loss of informationVSEase of operation

Solution Approach 1:

Instead of using abstract virtual representations of obstacles, the patent captures actual images of the real world including hands and objects, and displays these photographic copies on the HMD screen, providing direct visual information about real-world objects and enabling users to perform hands-on tasks while maintaining VR immersion

Inventive Principle:

Principle #26Copying

Data Source

PatentUS11030821B2Image display control apparatus and image display control program

Publication Date: 2021.06.08 ALPHA CODE INC

AI summary

A target object detecting unit 13 that detects a target object existing within a predetermined distance from an HMD 200 from a moving image of a real world captured by a camera 202 installed in the HMD 200 displaying a virtual space image and an image superimposition unit 15 that causes an image of a predetermined range including the target object to be displayed superimposed on a virtual space image are provided, and when a user performs a predetermined work with a hand within a range photographed in the camera 202 installed in the HMD 200, the captured image of the predetermined range including the hand is displayed superimposed on the virtual space image, and thus the user can appropriately perform the predetermined work while looking at the captured image displayed superimposed on the virtual space image even while wearing the HMD 200.