Remote Machine Tool Guidance Using Visual Tags for Depth Perception
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Remote operation of working machines like excavators and loaders lacks depth perception, making it difficult for operators to control tools like buckets with loose material efficiently and safely due to 2D image limitations.
Innovation Solution
A system using visual tags on load carriers to provide distance information, combined with sensor data, to enhance depth perception and enable autonomous or semi-autonomous operation assistance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If standard 2D camera feeds are used for remote operation, then the system complexity is low, but depth perception and spatial awareness are insufficient
Solution Approach 1:
The patent introduces visual tags as intermediary objects placed on load carriers. These tags serve as mediators between the 2D camera system and the 3D spatial understanding requirement, enabling depth perception without complex 3D sensing hardware. The tags provide reference points that the system can use to calculate distances and spatial relationships.
Solution Approach 2:
The patent transforms 2D image data into 3D spatial information by introducing visual tags with known dimensions and positions. By detecting the size and position of these tags in 2D images, the system infers 3D distance and spatial relationships, effectively adding a dimensional interpretation layer to the standard 2D camera feeds.
2Reliability
If visual tags and distance calculation systems are added, then depth perception and operational safety are improved, but the system complexity increases
Solution Approach 1:
The visual tags are self-contained reference objects that provide their own measurement information. They don't require active sensors, power sources, or complex communication systems on the load carrier side. The tags passively provide visual information that the remote system can process, making the safety enhancement self-service rather than requiring bidirectional complex systems.
Solution Approach 2:
The patent replaces potential mechanical measurement systems (such as physical distance sensors, LIDAR, or ultrasonic sensors) with an optical-based visual tag system. This substitution uses image processing and geometric calculation instead of mechanical or electronic ranging devices, simplifying the overall system while maintaining measurement capability.
3Measurement precision
If 2D camera feeds are used without additional sensors, then the device complexity is low, but the measurement precision of distance is insufficient
Solution Approach 1:
The patent changes the interpretive parameters of the camera system by introducing tags with known physical dimensions. Instead of trying to directly measure distance from raw pixel data, the system uses the known size of visual tags as a reference parameter to calculate real-world distances, transforming the camera from a simple imaging device into a measurement tool.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system and a method for remote operation of a working machine comprising a tool is provided. The system comprises an on-board controller configured to receive signals from a remote control station remotely controlling the operation of the working machine, and to obtain and send camera images to an off-board controller. The system also comprises an off-board controller configured to receive the camera images from the on-board controller. The off-board controller identifies at least one visual tag in the camera images located on a load carrier, determines at least one distance between the tool and the load carrier based on the identified at least one visual tag, and provides, to the on-board controller and/or to an operator of the working machine, information based on the determined at least one distance between the tool and the load carrier in order to support the remote operation of the working machine.