Passenger Gaze Object Recognition for In-Vehicle Information

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing information providing systems require passengers to manually point at targets to obtain information, lacking user-friendliness.

Innovation Solution

An information providing system that uses an in-vehicle terminal and a server apparatus to capture images, analyze voice inputs, and utilize machine learning models to identify areas of interest and recognize objects based on passenger gaze, providing relevant information without manual pointing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual pointing operation is required to obtain target information, then the system can accurately identify the target, but the ease of operation deteriorates

Engineering Contradiction:

Improvetarget identification accuracyVSAvoidoperational convenience

Core Design Contradiction:

Measurement precisionVSEase of operation

Solution Approach 1:

The patent replaces the mechanical manual pointing operation with an optical/imaging system. The imaging device captures images of the passenger's hand and finger gestures, and the control unit automatically detects and identifies targets based on these images, eliminating the need for direct mechanical pointing while maintaining identification accuracy.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system enables self-service by automatically detecting hand gestures and identifying targets without requiring the passenger to perform complex manual operations. The control unit autonomously processes the captured images, detects finger pointing directions, and retrieves target information, allowing the passenger to simply point naturally.

Inventive Principle:

Principle #25Self-service

2Reliability

If the system processes all captured images to identify targets, then comprehensive target detection is achieved, but the processing load increases

Engineering Contradiction:

Improvetarget detection completenessVSAvoidprocessing load

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The patent applies local quality by focusing processing only on specific regions of the captured images where hand gestures are likely to occur. The control unit identifies and processes only the relevant portions of the image containing the passenger's hand and finger, rather than analyzing the entire image, thereby reducing processing load while maintaining detection reliability.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The system extracts and processes only the essential information from the captured images - specifically the hand gesture patterns and finger directions. By extracting only the relevant features needed for target identification and discarding unnecessary image data, the system reduces processing load while maintaining comprehensive target detection.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentEP4095490B1Information provision device, information provision method, information provision program, and recording medium

Publication Date: 2026.03.04 PIONEER IP

Figure 1
Figure 2
Figure 3

AI summary

An information providing apparatus 3 includes: an image obtaining unit 323 that obtains a captured image having, captured therein, surroundings of a moving body; an area extracting unit 324 that extracts an area of interest on which a line of sight is focused in the captured image; an object recognizing unit 325 that recognizes an object included in the area of interest in the captured image; and an information providing unit 326 that provides object information related to the object included in the area of interest.