Input Device Motion Sensing via Imaging and Acceleration Fusion

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing input devices face challenges in accurately sensing motion due to the complexity of acceleration detection, especially when moving in simple motions like swinging, as the acceleration data is often complicated, making it difficult to analyze the motion accurately using only motion sensors like acceleration sensors.

Innovation Solution

The integration of imaging means and motion detecting means, such as acceleration sensors and gyro-sensors, in the input device, which captures images of predetermined targets to determine motion, allowing for more accurate sensing by combining data from both methods, including the use of acceleration sensors to detect linear and rotational motions and gyro-sensors to directly detect angular velocity.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If only motion sensors like acceleration sensors are used to detect input device motion, then the device structure remains simple, but the motion sensing accuracy deteriorates due to the complexity of acceleration data analysis

Engineering Contradiction:
Improvemotion sensing accuracyVSAvoiddevice structure
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent combines imaging means (camera) and motion detecting means (acceleration sensor, gyro sensor) into a single input device. The imaging means captures images of imaging targets to determine device position and orientation, while the motion sensors provide complementary motion data. This merging allows the system to overcome the limitations of using either sensor type alone, achieving more accurate motion sensing by fusing data from multiple sources.

Inventive Principle:
Principle #5Merging (Combining)

2Measurement precision

If acceleration sensors are used to detect motion, then the device can sense general motion, but the accuracy deteriorates when analyzing simple motions like swinging due to complicated acceleration patterns

Engineering Contradiction:
Improvemotion analysis accuracyVSAvoidmotion detection complexity
Core Design Contradiction:
Measurement precisionVSDifficulty of detecting and measuring

Solution Approach 1:

The patent introduces imaging means as an intermediary to capture visual information about the input device's position and orientation relative to imaging targets. This intermediary provides a different measurement approach that is not subject to the same complexities as acceleration sensors when detecting simple swinging motions. The image data serves as a mediator that complements the acceleration sensor data, allowing for more accurate motion analysis.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces part of the mechanical sensing system (acceleration sensors) with an optical sensing system (imaging means). Instead of relying solely on mechanical acceleration sensors to detect motion, the system uses optical capture of imaging targets to determine device position and orientation. This substitution eliminates the complexity of interpreting acceleration patterns for simple motions while maintaining the ability to detect device movement.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

3Measurement precision

If imaging means and motion detecting means are integrated in the input device, then motion sensing accuracy improves, but the device complexity increases

Engineering Contradiction:
Improvemotion sensing accuracyVSAvoiddevice structure
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent designs the input device with multi-functional components. The imaging means serves multiple purposes: determining device position, orientation, and motion relative to imaging targets. The motion detecting means (acceleration and gyro sensors) also provide multiple functions including detecting linear acceleration, rotational motion, and serving as backup when imaging targets are not visible. This multi-functionality justifies the increased device complexity by providing robust, accurate motion sensing under various conditions.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This combination enables more precise determination of the input device's motion, improving accuracy and reliability in sensing both linear and rotational movements, even in complex scenarios, by utilizing image-based calculations and sensor data to correct and enhance motion detection.

Implementation Method 1

imaging means for capturing predetermined imaging targets and motion detecting means for detecting a motion of the input device

Methodology Applied
Scientific EffectImage capture and analysis: Photography

Implementation Method 2

motion detecting means for detecting a motion of the input device including imaging means for capturing predetermined imaging targets

Methodology Applied
Scientific EffectAcceleration detection: Accelerometer

Implementation Method 3

the use of acceleration sensors to detect linear and rotational motions and gyro-sensors to directly detect angular velocity

Methodology Applied
Scientific EffectAngular velocity detection: Gyroscope

Data Source

PatentEP2016984B1Computer-readable storage medium having stored therein information processing program and information processing apparatus
Publication Date: 2019.03.27 NINTENDO CO LTD
  • EP2016984B1 patent drawingFigure 1
  • EP2016984B1 patent drawingFigure 2
  • EP2016984B1 patent drawingFigure 3~4

AI summary

A controller includes imaging means for capturing predetermined imaging targets and acceleration detecting means for detecting an acceleration applied to an input device. A game apparatus repeatedly calculates a target position which is related to images, included in a captured image captured by the imaging means, of the imaging targets and which is provided in the captured image, and then senses a predetermined motion (amotion in an upward direction) of the input device based on the target position calculated within a predetermined time period. Further, the game apparatus senses the predetermined motion of the input device based on the acceleration detected by the acceleration detecting means. Furthermore, the game apparatus executes a predetermined process (a process of causing a player character to perform a jump) when the predetermined motion has been sensed in at least either one of a first motion determining step and a second motion determining step.