Vehicle Gesture Input Using Multi-Camera Fusion for Detection Accuracy

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing manipulation input devices for automotive vehicles face challenges in accurately detecting hand gestures due to varying ambient environments and an increasing number of apparatuses and operational objects, leading to user burden and reduced detection accuracy.

Innovation Solution

A manipulation input device comprising multiple picture-taking apparatuses, hand region detectors, and hand manipulation determination units that select a single result from multiple determinations, using visible light and infrared cameras to enhance detection accuracy and simplify gesture manipulation across different environments.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If multiple picture-taking apparatuses are used to improve detection accuracy in varying ambient environments, then detection reliability is improved, but device complexity increases

Engineering Contradiction:

Improvegesture detection accuracyVSAvoidnumber of cameras and processing units

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The system divides the detection task among multiple specialized picture-taking apparatuses (visible light camera and infrared camera), each optimized for specific ambient conditions. The hand region detector and manipulation determination units process images from different sensors separately before integrating results, allowing each component to be optimized independently while maintaining overall system reliability across varying environments.

Inventive Principle:

Principle #1Segmentation

2Adaptability or versatility

If the number of apparatuses and operational objects increases, then system functionality is improved, but the number of required hand gestures increases burdening the user

Engineering Contradiction:

Improvenumber of controllable apparatusesVSAvoiduser learning burden

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The system implements a universal set of hand manipulation gestures that can control multiple different apparatuses and operational objects. The hand manipulation determination units recognize a standardized vocabulary of gestures (such as pointing, grasping, waving) that maintain consistent meaning across different contexts and target objects, allowing users to interact with increasingly complex systems without learning new gesture sets for each function.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Device complexity

If gesture detection relies on single camera images, then device complexity is reduced, but detection accuracy decreases in varying ambient environments

Engineering Contradiction:

Improvenumber of sensorsVSAvoidgesture detection accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The system merges information from multiple picture-taking apparatuses operating in different spectral ranges (visible light and infrared). The hand region detector combines detection results from both camera types, and the manipulation determination units integrate motion analysis from multiple sources. This fusion of data from different sensing modalities compensates for individual sensor limitations in varying ambient conditions, maintaining high detection accuracy without requiring an excessive number of sensors.

Inventive Principle:

Principle #5Merging (Combining)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

The device achieves high detection accuracy and simplifies gesture manipulation by using multiple cameras to select a single result, reducing the number of hand motions and shapes required, even in diverse ambient conditions.

Implementation Method 1

picture-taking apparatus for taking a picture of the user

Methodology Applied

Scientific EffectVisible light reflection: Reflection

Implementation Method 2

using visible light and infrared cameras to enhance detection accuracy

Methodology Applied

Scientific EffectInfrared radiation detection: Infrared Radiation

Data Source

PatentUS8681099B2Manipulation input device which detects human hand manipulations from captured motion images

Publication Date: 2014.03.25 MITSUBISHI ELECTRIC CORP

AI summary

When a vehicle navigation system is manipulated by taking pictures of a user hand motion and gesture with a camera, as the number of apparatuses and operational objects increases, the associated hand shapes and hand motions increase, thus causing a complex manipulation for a user. Furthermore, in detecting a hand with the camera, when the image of a face having color tone information similar to that of a hand appears in an image taken with a camera, or outside light rays such as sun rays or illumination rays vary, detection accuracy is reduced. To overcome such problems, a manipulation input device is provided that includes a limited hand manipulation determination unit and a menu representation unit, whereby a simple manipulation can be achieved and manipulation can accurately be determined. In addition, detection accuracy can be improved by a unit that selects a single result from results determined by a plurality of determination units, based on images taken with a plurality of cameras.