Remote Working-Machine Control With Delay-Compensated Indicator Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing remote operation systems face issues with deviation or disappearance of indicator images, such as line images, due to transmission failures or position deviations when operating a working machine, which affects the accuracy and efficiency of remote operation.
Innovation Solution
A remote operation device that includes a remote control element to recognize operation forms and transmit command signals, a first output control element to output a work environment image, and a second output control element to estimate and superimpose an indicator image representing the working unit's space occupying form without spoiling visibility, using arithmetic processing to account for communication and response delays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If a line image is created and transmitted from the working machine to reduce image delay, then the operator can perform operation while complementing image delay, but when transmission fails the line image disappears or position deviation increases
Solution Approach 1:
The patent introduces a position estimation function as an intermediary mechanism that calculates the working unit's position based on operation history and mechanical parameters when direct image transmission fails. This mediator maintains position information continuity by using alternative data sources (operation commands, mechanical state) to estimate position, ensuring the indicator image remains reliable even when the primary transmission path fails.
Solution Approach 2:
The system performs preliminary position estimation by continuously tracking operation forms and mechanical parameters before transmission failures occur. The position estimation function pre-calculates expected positions based on operation history, so when transmission fails, the system already has prepared position data to maintain the indicator image without interruption or deviation.
2Measurement precision
If the line image is superimposed on the photographed video to show working unit position, then the operator can accurately control the working unit, but transmission failure causes position deviation that reduces control accuracy
Solution Approach 1:
The patent implements a feedback mechanism where the position estimation function continuously monitors operation forms and mechanical parameters to update the indicator image position. This feedback loop ensures that when transmission fails, the system uses real-time operational data to maintain accurate position representation, preventing deviation and preserving measurement precision.
Solution Approach 2:
The system prepares compensatory position estimation mechanisms in advance to cushion against transmission failures. By having the position estimation function ready and continuously operating, the system creates a safety buffer that prevents position deviation when primary transmission fails, maintaining measurement precision through pre-prepared alternative positioning methods.
3Loss of time
If an indicator image is superimposed to show working unit position, then the operator can compensate for image delay, but the indicator image may spoil visibility of the working unit
Solution Approach 1:
The patent applies local quality by making the indicator image semi-transparent rather than fully opaque. This allows the indicator to provide position information while simultaneously permitting the underlying working unit image to remain visible through the indicator, thus compensating for time delay without completely blocking the view of the actual working unit.
Solution Approach 2:
The system uses color and transparency variations in the indicator image to balance information display and visibility. By adjusting the indicator's visual properties (transparency, color intensity), the system maintains the compensatory function for image delay while preserving the working unit's visibility, allowing operators to see both the indicator position and the actual working unit simultaneously.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
To provide a remote operation device and a remote operation system that can reduce or eliminate deviation of the position of an indicator image of a working unit such as a line image in an image with respect to the position of the working unit in a real space. In a remote operation device 20, an indicator image Img2 is generated and, then, superimposed on a work environment image Img1 and output to an image output device 221. In some case, a position posture coordinate value of a working unit corresponding to captured image data in a work environment image discontinuously changes because of a cause such as a communication failure between the remote operation device 20 and a working machine 40 due to a transmission delay of captured image data and deviates from a position posture coordinate value of the working unit estimated according to an operation form of a remote operation mechanism 211. In this case, a semitransparent indicator image Img2(τ2) simulating a bucket 445 is superimposed on a work environment image Img1(τ2) in a position where the bucket 445 is estimated to be originally present and deviating from an image region representing the bucket 445, and output to the image output device 221.