3D Map Display for Remote Vehicle Teleoperation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Operators controlling remote vehicles, such as those used in Explosive Ordnance Disposal, face cognitive overload due to limited situational awareness, relying on monocular camera feeds for detailed control, leading to increased mission execution time and risk exposure.
Innovation Solution
A system that provides a 3D map display around the remote vehicle using stereo vision data, visual odometry, and fill-in algorithms to estimate depth in texture-lacking areas, integrated with an operator control unit for enhanced situational awareness and intuitive control.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If monocular camera feeds are used for detailed control, then the operator can control the remote vehicle, but the operator experiences cognitive overload and lacks situational awareness
Solution Approach 1:
The system segments the visual information into two distinct components: a first video feed providing detailed monocular view for control tasks, and a second video feed providing wide-angle situational awareness. This segmentation allows the operator to access both detailed control information and broader environmental context simultaneously, resolving the contradiction between control capability and situational awareness.
Solution Approach 2:
The system introduces a dual-video-feed interface as an intermediary between the operator and the remote vehicle environment. This intermediary presents two concurrent video streams that together provide both detailed control information and comprehensive situational awareness, eliminating the need for the operator to choose between the two.
2Measurement precision
If detailed control is required using limited sensory information, then the operator can precisely control the remote vehicle, but the mission execution time increases
Solution Approach 1:
The control interface is segmented into two video feeds with different fields of view. The first video feed maintains the detailed monocular view necessary for precise control operations, while the second video feed provides wide-angle context to reduce navigation time and improve overall mission efficiency by eliminating the need for frequent camera repositioning.
3Loss of information
If the operator focuses on down range surroundings, then situational awareness of the environment is improved, but the operator becomes unaware of events in the immediate environment
Solution Approach 1:
The video feed is segmented into two concurrent streams: one maintaining a narrow, detailed view for precise control and environmental inspection, and the other providing a wide-angle view that captures both the down-range environment and the immediate surroundings. This allows the operator to monitor both distant and nearby events simultaneously, eliminating the safety risk of being unaware of immediate environmental changes.
Data Source
AI summary
A method for improving situational awareness for teleoperation of a remote vehicle by creating a 3D map display of an area around the remote vehicle comprises: receiving an original image from a stereo vision camera and utilizing the original image to perform visual odometry to determine the x, y, z, roll, pitch, and yaw for the original image; applying a fill-in algorithm to the original image to fill in an estimated depth for areas of the original image for which no depth data is available, which creates an enhanced depth image; combining the enhanced depth image with the x, y, z, roll, pitch, and yaw for the original image to create the 3D map display of the area around the remote vehicle; and displaying the 3D map display on an operator control unit used to control the remote vehicle.