Hybrid Position Tracking Using Camera and Sensor Fusion

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional input systems using camera-based marker tracking limit user movement to the camera's field of view, restricting operability and accuracy in input operations.

Innovation Solution

An information processing system that calculates a designated position by combining imaging information and sensor data, allowing the input device to be used outside the camera's field of view by determining a first position based on imaging and a second position based on sensor data, and switching between these positions to enhance operability and accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If camera-based marker tracking is used to calculate input position, then measurement precision is improved, but ease of operation deteriorates due to limited field of view

Engineering Contradiction:
Improveposition calculation accuracyVSAvoidinput device movement range
Core Design Contradiction:
Measurement precisionVSEase of operation

Solution Approach 1:

The patent merges two different measurement approaches: camera-based imaging information and sensor-based detection results. The position calculation section combines both first position (from imaging) and second position (from sensor) data to determine the final designated position, allowing the system to leverage the precision of camera tracking while maintaining the operational freedom of sensor-based movement detection.

Inventive Principle:
Principle #5Merging (Combining)

2Ease of operation

If sensor-based position calculation is used, then ease of operation is improved, but measurement precision deteriorates

Engineering Contradiction:
Improveinput device movement rangeVSAvoidposition calculation accuracy
Core Design Contradiction:
Ease of operationVSMeasurement precision

Solution Approach 1:

The patent introduces an intermediary position calculation mechanism that bridges sensor data and imaging data. The position calculation section acts as an intermediary that processes both sensor-based second position and camera-based first position, combining them to produce a final designated position that inherits advantages from both sources while mitigating their individual weaknesses.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Measurement precision

If only imaging information is used for position calculation, then measurement precision is improved, but adaptability deteriorates when marker is out of view

Engineering Contradiction:
Improveposition calculation accuracyVSAvoidoperational flexibility
Core Design Contradiction:
Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent changes the operational parameters by switching between different position sources based on visibility conditions. When the marker is visible, the system uses imaging information (first position) for high precision. When the marker is out of view, the system transitions to using sensor detection results (second position), maintaining operational flexibility while adapting to changing visibility conditions.

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS9354706B2Storage medium having stored therein information processing program, information processing apparatus, information processing system, and method of calculating designated position
Publication Date: 2016.05.31 NINTENDO CO LTD
  • US9354706B2 patent drawing
  • US9354706B2 patent drawing
  • US9354706B2 patent drawing

AI summary

An example of an information processing apparatus calculates a first position using imaging information based on an image captured by an imaging device. The information processing apparatus calculates a second position using a detection result of a sensor. The information processing apparatus repeatedly determines whether or not a predetermined condition is satisfied, the predetermined condition including a condition regarding a relationship between the first position and the second position. A designated position on a predetermined plane is calculated using the first position and/or the second position. In a satisfaction period after the predetermined condition has been satisfied, the information processing apparatus calculates the designated position by making an influence of the second position relatively great as compared to an unsatisfaction period before the predetermined condition is satisfied.