Eyeball Movement Sensor for Operator Action Boundary Detection

Solution Overview

Problem

Existing methods for capturing work performed by an operator with an image capturing device attached to the operator struggle to distinguish actions accurately due to the identification marker not always being within the capturing region, leading to difficulties in determining action boundaries.

Innovation Solution

An information processing apparatus that includes a processor to acquire moving images, detect eyeball movements using sensors, and specify action boundaries based on predetermined conditions such as blinking or fixation states, allowing for accurate decomposition of work into actionable parts.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If an image capturing device attached to an operator is used to capture work, then the operator's work can be captured from their perspective, but the identification marker may not be included in the capturing region, making it difficult to distinguish actions

Engineering Contradiction:

Improvecapturing work from operator's perspectiveVSAvoidaction boundary detection accuracy

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent introduces eyeball movement detection as an intermediary mechanism to bridge the gap between the operator's perspective capture and action boundary detection. Instead of relying solely on identification markers that may be out of frame, the system uses eye tracking data as a mediator to accurately determine when actions occur, thereby maintaining both the operator's perspective viewpoint and precise action boundary detection.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces the mechanical/visual approach of using identification markers visible in the image capture region with a physiological sensing approach. By substituting the marker-based visual detection system with an eyeball movement sensor system, the patent eliminates the limitation of markers being out of frame while maintaining the ability to precisely detect action boundaries through eye movement patterns.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Measurement precision

If identification markers are used to detect action boundaries, then action boundaries can be determined, but the markers are not always visible in the capturing region when the device is attached to the operator

Engineering Contradiction:

Improveaction boundary detection accuracyVSAvoidmarker visibility reliability

Core Design Contradiction:

Measurement precisionVSReliability

Solution Approach 1:

The patent introduces eyeball movement detection as an intermediary mechanism to bridge the gap between the operator's perspective capture and action boundary detection. Instead of relying solely on identification markers that may be out of frame, the system uses eye tracking data as a mediator to accurately determine when actions occur, thereby maintaining both the operator's perspective viewpoint and precise action boundary detection.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces the mechanical/visual approach of using identification markers visible in the image capture region with a physiological sensing approach. By substituting the marker-based visual detection system with an eyeball movement sensor system, the patent eliminates the limitation of markers being out of frame while maintaining the ability to precisely detect action boundaries through eye movement patterns.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS12061736B2Information processing apparatus

Publication Date: 2024.08.13 FUJIFILM BUSINESS INNOVATION CORP

AI summary

An information processing apparatus includes: a processor configured to: acquire a moving image captured by an image capturing device attached to an operator; detect an eyeball movement of the operator using a sensor attached to the operator; and specify, from among the acquired moving image, a portion that was captured when a speed of the detected eyeball movement satisfied a predetermined condition as a boundary between actions of the operator.