Gaze-Guided Work Estimation for Multi-Object Contact Recognition

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing behavior recognition methods inaccurately recognize user actions when multiple objects are present in the user's view, leading to erroneous decisions about user work, especially when tools are not being used.

Innovation Solution

A work estimation device that includes a watching region estimation unit, an object detection unit, and a contact degree calculation unit to determine the degree of contact between a work object and candidate objects based on the user's line of sight, allowing for precise work estimation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Extent of automation

If behavior recognition is performed based on object detection in user viewpoint video, then work estimation can be automated, but erroneous recognition occurs when multiple objects are present

Engineering Contradiction:
Improveautomated work estimationVSAvoidbehavior recognition accuracy
Core Design Contradiction:
Extent of automationVSMeasurement precision

Solution Approach 1:

The patent introduces an attention map as an intermediary element that mediates between the raw video input and the behavior recognition output. The attention map highlights regions of interest in the video frame, allowing the system to focus on relevant objects while ignoring irrelevant ones. This intermediary mechanism enables automated work estimation while preventing erroneous recognition caused by multiple objects in the field of view.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent applies local quality by making different regions of the video frame have different recognition weights. Instead of treating all detected objects equally, the system assigns higher importance to objects within the attention map's highlighted regions and lower importance to objects outside these regions. This local differentiation allows accurate behavior recognition even when multiple objects are present in the video.

Inventive Principle:
Principle #3Local quality

2Adaptability or versatility

If all detected objects are considered as work targets, then comprehensive work coverage is achieved, but false positive recognition increases

Engineering Contradiction:
Improvework target coverageVSAvoidrecognition reliability
Core Design Contradiction:
Adaptability or versatilityVSReliability

Solution Approach 1:

The attention map serves as a filtering intermediary between object detection and work target identification. It allows the system to maintain comprehensive object detection capability while reliably distinguishing actual work targets from background objects or irrelevant items by highlighting only the regions that require attention.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The system applies different quality weights to different detected objects based on their location within the attention map. Objects falling within the attention map's highlighted regions are treated as high-priority work targets, while objects outside these regions are treated as low-priority or non-work targets, thereby improving recognition reliability without reducing adaptability.

Inventive Principle:
Principle #3Local quality

3Device complexity

If behavior recognition uses simple object presence detection, then system complexity is reduced, but estimation precision deteriorates

Engineering Contradiction:
Improverecognition system complexityVSAvoidwork estimation precision
Core Design Contradiction:
Device complexityVSMeasurement precision

Solution Approach 1:

The attention map introduces a moderate level of complexity as an intermediary processing step between simple object detection and final behavior recognition. While this adds some computational overhead, it dramatically improves estimation precision by enabling the system to distinguish relevant from irrelevant objects without requiring complex mechanical or structural modifications.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces complex mechanical or manual verification systems with an attention-based computational mechanism. Instead of using elaborate physical sensors or manual confirmation processes to identify work targets, the system uses attention maps generated through image processing algorithms, achieving high precision with relatively simple computational infrastructure.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS12525060B2Work estimation device, work estimation method, and non-transitory computer readable medium
Publication Date: 2026.01.13 MITSUBISHI ELECTRIC CORP
  • US12525060B2 patent drawing
  • US12525060B2 patent drawing
  • US12525060B2 patent drawing

AI summary

A work estimation device (200) includes a watching region estimation unit (220), an object detection unit (210), a contact degree calculation unit (230), and a work estimation unit (250). The watching region estimation unit (220) estimates a watching region which is a region a user watches, with using information indicating a line of sight of the user. The object detection unit (210) detects, from a video that shows a work object the user is using and at least one target candidate object which is a candidate for a work target of the user, the work object and the at least one target candidate object. The contact degree calculation unit (230) calculates a contact degree indicating a degree of contact between the work object and each target candidate object which is included in the at least one target candidate object, on a basis of the watching region. The work estimation unit (250) estimates work of the user on a basis of the work object and the contact degree.