Controller Orientation Correction in xR Systems

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In xR systems, users face difficulties in recognizing the orientation of controllers, especially when the display device is worn on the head, leading to inaccurate control of virtual objects due to obstacles blocking the marker detection between the controller and the camera.

Innovation Solution

An information processing device and method that acquires operation information from a two-dimensional operation on a controller and corrects it based on the correspondence relationship between the operation coordinate system and the display coordinate system, allowing for precise control of virtual objects displayed on a display device.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If a camera-based marker detection system is used to specify controller position, then the virtual object can be controlled based on detected position, but the control fails when the marker or camera is blocked by obstacles

Engineering Contradiction:

Improvecontroller position detection reliabilityVSAvoidmarker detection system complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent replaces the optical marker detection system with an inertial measurement system. Instead of using a camera to detect a marker on the controller, the system uses accelerometers and gyroscopes to detect the controller's orientation and position changes. This substitution eliminates the need for line-of-sight between camera and marker, resolving the blocking issue while maintaining position detection capability.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent introduces an intermediary coordinate system transformation mechanism. It establishes a correspondence relationship between the controller's operation coordinate system and the display device's display coordinate system, using transformation matrices to map operation directions to display positions. This intermediary transformation layer enables accurate control despite orientation differences between devices.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Ease of operation

If the display device is worn on the head to enable immersive viewing, then the user experience is improved, but the user's field of vision is blocked making it difficult to recognize controller orientation

Engineering Contradiction:

Improveuser interaction convenienceVSAvoidcontroller orientation recognition difficulty

Core Design Contradiction:

Ease of operationVSDifficulty of detecting and measuring

Solution Approach 1:

The patent replaces visual orientation recognition with inertial sensing. Instead of relying on the user to visually recognize controller orientation (which is difficult when wearing a head-mounted display), the system uses accelerometers and gyroscopes to automatically detect the controller's orientation in three-dimensional space. This substitution removes the visual recognition requirement entirely.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent implements feedback by continuously tracking the controller's orientation changes through inertial sensors and automatically adjusting the virtual object's movement direction accordingly. The system provides real-time feedback between the controller's physical orientation and the virtual object's movement, eliminating the need for the user to manually recognize or adjust for orientation differences.

Inventive Principle:

Principle #23Feedback

3Adaptability or versatility

If the controller and display device have different orientations, then each device can be used in its optimal orientation, but the operation direction does not align with the display position

Engineering Contradiction:

Improvedevice orientation flexibilityVSAvoidcontrol accuracy

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent extends the coordinate transformation from two-dimensional screen coordinates to three-dimensional spatial coordinates. By incorporating the third dimension (depth/vertical axis) into the coordinate system transformation, the system can accurately map controller movements in any orientation to the appropriate display positions, maintaining precision across multiple orientation dimensions.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS12118155B2Information processing device and information processing method

Publication Date: 2024.10.15 CANON KK

AI summary

An information processing device includes at least one memory and at least one processor which function as: an acquisition unit configured to acquire operation information of a two-dimensional operation performed on a controller; and a control unit configured to control an object displayed on a display device based on the operation information and a correspondence relationship between an operation coordinate system for determining an operation direction of the two-dimensional operation performed on the controller and a display coordinate system for determining a display position of the display device.