Tablet Position Overlay in HMDs Using Code Recognition

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Users wearing a head-mounted display (HMD) cannot accurately recognize the position of a tablet due to low accuracy in specifying the tablet's position, especially when moving, which affects their ability to interact with the tablet effectively.

Innovation Solution

An output control system that utilizes an imaging device to recognize codes displayed on a tablet in the real space, allowing for precise specification of the tablet's position, and then displays a shape representing the input region of the tablet in the virtual space within the HMD, enhancing accuracy and usability.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If infrared rays are used to specify the position of the tablet, then the user can recognize the tablet position, but the accuracy of specifying the position is low

Engineering Contradiction:
Improveposition specification accuracyVSAvoiduser ability to recognize tablet position
Core Design Contradiction:
Measurement precisionVSEase of operation

Solution Approach 1:

The patent uses an imaging device to capture an image of the tablet and its surroundings, then generates a depth map by comparing the captured image with a reference image. This optical copying approach replaces the inaccurate infrared position specification method, achieving high-precision position detection through image processing rather than direct infrared measurement.

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The patent substitutes the infrared-based position specification system with an optical imaging and image processing system. By using an imaging device to capture visual information and processing the images to generate depth maps, the system achieves more accurate position detection without relying on infrared radiation and its associated limitations.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

2Adaptability or versatility

If a head mounted display is used, then the user can view virtual space, but the user cannot see the tablet in real space

Engineering Contradiction:
Improvevirtual space display capabilityVSAvoidreal space tablet visibility
Core Design Contradiction:
Adaptability or versatilityVSEase of operation

Solution Approach 1:

The patent introduces an imaging device as an intermediary between the user and the tablet. The imaging device captures images of the tablet and the user's hand, and the system processes these images to provide position information and visual feedback in the virtual space displayed by the HMD, effectively mediating the user's interaction with the tablet without requiring direct visual contact.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The system creates visual copies of the tablet and hand movements in the virtual space through image processing. By capturing real-world images and rendering them in the virtual environment, the user can see representations of physical objects and actions in the HMD display, maintaining awareness of real-space interactions while wearing the head-mounted display.

Inventive Principle:
Principle #26Copying

3Measurement precision

If code recognition is used to specify tablet position, then position accuracy is improved, but additional imaging processing is required

Engineering Contradiction:
Improvetablet position accuracyVSAvoidimaging processing requirements
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The imaging device serves multiple functions: it captures images for position specification, tracks hand movements, provides visual feedback, and generates depth maps. By making the imaging device multi-functional, the patent avoids adding separate specialized devices for each function, thereby managing complexity while achieving high position accuracy through code recognition and image processing.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

Improves the accuracy of specifying the tablet's position, enabling users to interact more effectively with the tablet while wearing an HMD, even when moving, and reduces the need for additional infrared communication devices, thereby enhancing versatility and cost-effectiveness.

Implementation Method 1

recognizing a code displayed on the tablet in a real space using an imaging device

Methodology Applied
Scientific EffectImage recognition: Photography

Data Source

PatentUS12493438B2Output control system, output control method, and recording medium
Publication Date: 2025.12.09 NEC CORP
  • US12493438B2 patent drawing
  • US12493438B2 patent drawing
  • US12493438B2 patent drawing

AI summary

This output control system comprises: a specifying unit; and a first output control unit. The specifying unit specifies the position of a tablet by recognizing the code displayed on a tablet in a real space by using an imaging device. The first output control unit displays a shape representing an input area of the tablet of which the position is specified in a virtual space on the head-mounted display.