HMD Display Control Mode Switching for Cursor Positioning

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing mixed reality and virtual reality technologies using Head Mounted Displays (HMDs) do not effectively allow users to conveniently control multiple display items indicating positions, such as mouse cursors and selection frames, leading to difficulties in operability.

Innovation Solution

An information processing apparatus and method that includes a first display control unit for displaying a first display item based on inertial information, a second display control unit for displaying a second display item based on user operations, and a switching unit that transitions between control modes when the first display item's position enters a specific area, allowing for convenient control of multiple display items.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If a single control mode is used for displaying position-indicating display items, then the control logic is simple, but the user cannot conveniently switch between different control methods (e.g., inertial control vs. operation-member control)

Engineering Contradiction:
Improvecontrol mode flexibilityVSAvoidcontrol system complexity
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system dynamically switches between first control mode (inertial information-based) and second control mode (operation member-based) depending on the situation. The switching unit changes the control mode according to whether the display item position is in the first area or second area, making the control system adaptive rather than static.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The information processing apparatus is designed to handle multiple control modes within a single system. The display control unit can operate in either first control mode or second control mode, allowing the same system to serve multiple control purposes without requiring separate dedicated systems.

Inventive Principle:
Principle #6Universality (Multi-functionality)

2Ease of operation

If multiple display items are controlled in a single control mode, then the control logic remains simple, but the user experience deteriorates when different control methods are needed for different display items

Engineering Contradiction:
Improvedisplay item control convenienceVSAvoidcontrol mode management complexity
Core Design Contradiction:
Ease of operationVSDevice complexity

Solution Approach 1:

Different control modes are applied to different spatial areas. The first control mode is used when the display item is in the first area, while the second control mode is used when the display item is in the second area. This local differentiation optimizes control convenience for each specific situation without requiring complex global control logic.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The control mode switches dynamically based on the display item's position and the user's needs. The switching unit automatically transitions between control modes, eliminating the need for users to manually manage complex control mode settings while still providing access to multiple control methods.

Inventive Principle:
Principle #15Dynamics

3Area of stationary object

If the display field of view is expanded in HMDs, then the user can see more content, but the user loses sight of display items indicating positions more easily

Engineering Contradiction:
Improvedisplay field of viewVSAvoidloss of sight of display items
Core Design Contradiction:
Area of stationary objectVSLoss of information

Solution Approach 1:

The display field is segmented into different areas (first area and second area), each associated with different control modes. This segmentation allows the system to apply appropriate control strategies for different regions, helping users maintain awareness of display item positions even in an expanded field of view.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The system provides feedback to the user about the display item's position and control mode status. By switching control modes based on position, the system helps users maintain spatial awareness and prevents loss of sight of display items through adaptive control adjustments.

Inventive Principle:
Principle #23Feedback

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

Enables users to conveniently control multiple display items, improving operability by switching between control modes based on user interactions, thereby preventing loss of sight of display items in the vast field of view of HMDs.

Implementation Method 1

an attitude sensor unit which acquires inertial information

Methodology Applied
Scientific EffectInertial measurement: Accelerometer

Data Source

PatentUS12032754B2Information processing apparatus, information processing method, and non-transitory computer readable medium
Publication Date: 2024.07.09 CANON KK
  • US12032754B2 patent drawing
  • US12032754B2 patent drawing
  • US12032754B2 patent drawing

AI summary

An information processing apparatus performs control, in a first control mode, so that a first display item is displayed on a display region of a display unit, the first display item indicating a position based on inertial information of a controller. The information processing apparatus performs control, in a second control mode, so that a second display item is displayed on the display region, the second display item indicating a position according to a first operation performed on a specified operation member in a first area of the display region. The information processing apparatus performs, in a case where the position indicated by the first display item is included in a second area of the display region in the first control mode, switching from the first control mode to the second control mode in response to a specified operation.