Dynamic Visual Overlay for Remote Construction Equipment Terrain Perception
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Remote operation of machines, such as construction equipment, faces challenges in terrain perception due to the lack of depth information in two-dimensional video feeds from cameras, which can be obscured by image compression and lens distortion, making it difficult for operators to understand terrain features, especially when the machine is stationary.
Innovation Solution
The implementation of a dynamic visual overlay system that uses 3D data from sensors to project depth information onto the video feed, allowing the overlay to move along dimensions of the image, enhancing terrain perception by providing semi-transparent bands representing depth ranges and colorizing based on terrain height.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If 2D video feed from camera is used for remote operation, then the operator can view areas around the machine, but depth information of terrain features is lost
Solution Approach 1:
The patent applies dimensionality change by projecting 3D terrain data onto the 2D video feed to create a dynamic overlay that restores depth perception. The overlay uses semi-transparent bands that move along the depth dimension, allowing remote operators to perceive terrain features' distance and elevation without adding physical 3D components to the camera system.
2Measurement precision
If the machine is stationary, then operational precision can be maintained, but terrain features become difficult to perceive without movement
Solution Approach 1:
The system performs preliminary action by pre-calculating and projecting 3D terrain data onto the video feed before the operator needs to make decisions. The dynamic overlay is continuously updated and positioned based on predicted terrain features, allowing operators to plan operations in advance even when the machine is stationary, eliminating the need for maneuvering to gather terrain information.
Solution Approach 2:
The patent introduces an intermediary element - the dynamic visual overlay - that mediates between the 2D camera feed and the operator's need for 3D terrain perception. This overlay acts as a bridge, translating depth information from 3D data into visual cues that enhance terrain feature perception without requiring the machine to move.
3Loss of information
If image compression and lens distortion are applied, then video transmission efficiency is improved, but terrain features become obscured
Solution Approach 1:
The patent applies local quality by enhancing only the specific regions of the video feed where terrain features are located, rather than processing the entire image uniformly. The dynamic overlay is selectively positioned and rendered based on detected terrain features, applying depth enhancement locally to relevant areas while leaving other regions unchanged, thus minimizing additional processing requirements.
Data Source
AI summary
During operation of a machine, an understanding of terrain features is critical. Accordingly, disclosed embodiments augment a video stream, captured by a camera mounted on the machine, with terrain information. In particular, an overlay is generated on at least one image frame. The overlay, which may comprise one or more semi-transparent bands, may cyclically recede from a foreground to a background of the image frame(s), cyclically advance from a background to a foreground of the image frame(s), or cyclically move horizontally across the image frame(s). The overlay may be colorized to illustrate the height of the terrain underneath the overlay.