Remote Machine Display With Gaze-Tracked Panoramic Imaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing display systems for remote control of working machines do not effectively track the operator's line of sight to adjust the displayed images, leading to suboptimal remote control operability.
Innovation Solution
A display system with multiple camera units, including a pair of left-eye and right-eye cameras, generates panoramic images and adjusts them based on the operator's eye and head movements, using an image processing device and remote control device to synchronize the images with the operator's gaze, and implements delay processes for rapid movements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If the display system uses conventional panoramic image display without line of sight tracking, then the system structure is simple, but the remote control operability is poor
Solution Approach 1:
The display system incorporates line of sight detection capability that provides feedback to the image processing device. The detection device detects the operator's line of sight direction, and this information is fed back to dynamically adjust and display the appropriate panoramic image portion, thereby improving remote control operability through adaptive image presentation.
Solution Approach 2:
The system replaces manual mechanical adjustment of panoramic images with automated optical/electronic line of sight detection. Instead of requiring operators to manually navigate through panoramic images, the system uses detection devices (such as eye trackers or head position sensors) to automatically determine which portion of the panoramic image should be displayed based on the operator's natural gaze direction.
2Ease of operation
If the display system tracks line of sight movement to change displayed images, then the remote control operability improves, but the response speed requirement increases
Solution Approach 1:
The system performs preliminary actions by continuously pre-processing panoramic images and maintaining multiple processed image portions ready for display. When line of sight detection occurs, the system can immediately switch to the pre-prepared corresponding image portion, reducing the response time required to adjust the displayed image in real-time.
Solution Approach 2:
The display system implements dynamic image adjustment that adapts to the operator's line of sight movement. The image processing device dynamically selects and displays different portions of the panoramic image based on detected line of sight direction, creating a responsive and adaptive display system that moves with the operator's gaze rather than requiring the operator to adapt to a static display.
3Loss of information
If the system transmits full panoramic images, then the image quality is high, but the wireless communication data transmission load increases
Solution Approach 1:
The system extracts and transmits only the necessary portions of panoramic images rather than complete panoramic images. The image processing device identifies and transmits only the specific image portions corresponding to the operator's line of sight direction, reducing wireless communication data transmission load while maintaining the completeness of relevant visual information for remote control operations.
Data Source
AI summary
The display system for remote control of a working machine includes a plurality of camera units and an image processing device, both provided on or in a working machine, and a display device and a remote control device, both provided in the remote control side. The display device detects movement of the head of an operator and transmits the detected movement of the head of the operator to the remote control device. The remote control device transmits, to the image processing device over wireless communications, the movement of the head of the operator transmitted from the display device. The image processing device adjusts, for transmission to the display device, parts of left-eye and right-eye panoramic images according to the movement of the head of the operator transmitted from the remote control device.