Remote Operation Gaze Overlay for Working Machine Action Guidance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Less skilled operators face challenges in understanding remotely operated actions of working machines, necessitating a system that provides appropriate information to enhance their grasp of the operations.
Innovation Solution
A remote operation system comprising a working machine with an imaging device and actuation mechanism, along with first and second remote operation apparatuses equipped with wireless communication, image output devices, and sight line detectors, which highlight designated image regions based on the operator's sight line to facilitate better understanding of the remotely operated actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If standard remote operation systems are used, then basic remote operation functionality is provided, but less skilled operators cannot grasp the remotely operated actions effectively
Solution Approach 1:
The patent applies color changes by displaying the sight line trajectory and designated image regions in distinct colors (e.g., blue for sight line, red for designated regions) against the environmental image background. This visual differentiation allows less skilled operators to immediately identify where the skilled operator is looking and what actions are being performed, enhancing their understanding without obscuring the actual environment
Solution Approach 2:
The patent introduces an intermediary information layer that mediates between the skilled operator's actions and the less skilled operator's understanding. By overlaying sight line data and designated region markers as intermediate visual elements, the system bridges the knowledge gap without requiring direct complex communication or additional training
2Ease of operation
If comprehensive information is provided to less skilled operators, then their understanding improves, but the system complexity increases
Solution Approach 1:
The patent implements universality by enabling each remote operation apparatus to perform multiple functions: the sight line detector in the first apparatus serves both to track the skilled operator's attention and to generate guidance information for the second apparatus. The second apparatus simultaneously receives environmental images and overlay guidance data, functioning as both a display terminal and a learning device
Solution Approach 2:
The patent applies copying by replicating the sight line detection and designated region marking functionality across multiple remote operation apparatuses. Each apparatus receives and displays copied versions of the environmental image with overlaid guidance information, allowing the less skilled operator to observe and replicate the skilled operator's actions in their own interface
Data Source
AI summary
Provided is a system which can supply appropriate information to one operator from a viewpoint that the one operator grasps a remotely operated action of a working machine by another operator. In a first remote operation apparatus 10, a sight line detector 112 detects a sight line of a first operator, and transmits sight line detection data corresponding to the sight line of the first operator. In a second remote operation apparatus 20, a second image output device 221 displays a designated image region spreading with reference to the sight line of the first operator corresponding to the sight line detection data in an environmental image corresponding to captured image data acquired by an imaging device 401 of a working machine 40, in a form different from a form of an image region around the designated image region.