Inertial Sensor and Image Detection for Game Control

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing information processing systems for mobile devices lack the capability to provide a variety of user operations, limiting the range and complexity of interactions, particularly in gaming applications.

Innovation Solution

An information processing system that utilizes inertial sensors and image capturing technology to detect user movements and orientations, allowing for a greater variety of user operations by adjusting detection conditions based on the position and shape of a detection target image and output data from inertial sensors, enabling more sophisticated game interactions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If traditional connectors and controllers are used, then the device structure is simple, but the variety of user operations is limited

Engineering Contradiction:

Improvevariety of user operationsVSAvoiddevice structure

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The accessory integrates multiple operation methods (physical buttons, motion sensing, image recognition) into a single device, allowing it to perform diverse functions including game control, gesture recognition, and visual interactions, thereby achieving multi-functionality without requiring multiple separate devices

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The accessory serves as an intermediary between the user and the information processing device, translating various user inputs (physical operations, motions, visual gestures) into commands that the device can process, thereby expanding operational versatility while keeping the core device structure unchanged

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If image capturing and inertial sensors are added, then user operation variety increases, but detection precision requirements increase

Engineering Contradiction:

Improveuser operation varietyVSAvoiddetection precision

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent combines image capturing section and inertial sensor data processing within the accessory itself, merging multiple detection modalities into a unified processing unit that can correlate visual and motion data to improve overall detection accuracy and reduce the precision burden on individual sensors

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The system processes detection target portion images and inertial sensor data to determine user operations, creating a feedback loop where the accessory continuously monitors both visual and motion inputs, adjusting its detection parameters based on the correlation between image position changes and inertial measurements to maintain precision across varied operation types

Inventive Principle:

Principle #23Feedback

3Measurement precision

If detection conditions are changed based on inertial sensor data, then operation recognition accuracy improves, but processing complexity increases

Engineering Contradiction:

Improveoperation recognition accuracyVSAvoidprocessing complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The accessory dynamically adjusts detection conditions based on real-time inertial sensor data, changing processing parameters such as image analysis thresholds and detection sensitivity according to the detected motion state, thereby improving operation recognition accuracy while adapting processing complexity to actual operational needs rather than maintaining fixed high complexity

Inventive Principle:

Principle #15Dynamics

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

Enables a wider range of user interactions and game control options by accurately detecting and responding to user movements and orientations, enhancing the gaming experience with improved responsiveness and complexity.

Implementation Method 1

data output from an inertial sensor

Methodology Applied

Scientific EffectInertial sensing: Accelerometer

Implementation Method 2

detecting a position and a shape of a detection target portion image

Methodology Applied

Scientific EffectGravity detection: Gravitation

Implementation Method 3

capturing a detection target portion image with an image capturing section

Methodology Applied

Scientific EffectLight reflection: Reflection

Data Source

PatentEP3513849B1Information processing system, information processing program, information processing method, and information processing apparatus

Publication Date: 2022.03.16 NINTENDO CO LTD

Figure 1
Figure 2
Figure 3

AI summary

A detection target portion of a device is captured by an image capturing section. An operation portion of the device is operated by a user and causes the detection target portion to move in conjunction with the operation portion in accordance with the operation. A processing section performs a first process that is information processing. A detection section detects a detection target portion image representing the detection target portion comprised in a captured image captured by the image capturing section. A specifying section specifies at least one of a position and a shape of the detection target portion image in the captured image. Based on the at least one of the position and the shape of the detection target portion image, a determination section determines whether or not a condition for performing a first process that is information processing is satisfied. Based on data output from an inertial sensor, the change section changes the condition.