VR Remote Vehicle Control for Hazardous Environment Takeover
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in navigating unknown or hazardous environments without direct human control, particularly at higher autonomy levels where no driver is present, as conventional remote control methods using 2D visualizations and generic computer controls lack immersion and natural control capabilities.
Innovation Solution
The implementation of virtual reality (VR) technology to create an immersive virtual environment for remote operators, allowing them to control autonomous vehicles or other objects through a VR headset, with sensor data encoding and calibration to provide intuitive and natural control inputs, such as steering and braking, mimicking real-world vehicle operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If 2D visualizations are used on 2D displays for remote control, then the system complexity is reduced, but the immersion and control safety are insufficient
Solution Approach 1:
The patent transitions from 2D display visualizations to a 3D virtual reality environment. The remote operator wears a VR headset that presents stereoscopic images and spatial audio, creating an immersive three-dimensional representation of the vehicle environment. This dimensional change provides depth perception and spatial awareness that 2D displays cannot achieve, thereby improving remote control safety while maintaining manageable system complexity through software-based VR rendering.
2Ease of operation
If generic computer components (keyboard, mouse, joystick) are used for control, then the ease of operation is improved, but the natural control capability and passenger comfort are reduced
Solution Approach 1:
The patent implements virtual copies of actual vehicle controls within the VR environment. A virtual steering wheel, accelerator pedal, and brake pedal are rendered in the VR scene, matching the physical vehicle's control layout and behavior. When the remote operator manipulates these virtual controls, the actions are transmitted to the actual vehicle, creating a natural one-to-one correspondence. This copying approach maintains ease of operation while ensuring passenger comfort through familiar control patterns.
3Productivity
If autonomous vehicles operate without direct human control, then the productivity is improved, but the ability to handle unknown environments is reduced
Solution Approach 1:
The patent introduces a remote operator as an intermediary between the autonomous vehicle and the environment. When the vehicle encounters an unknown or hazardous situation beyond its autonomous capabilities, control is transferred to a human operator who can be remotely present through the VR system. This intermediary approach allows the vehicle to maintain autonomous operation for routine tasks (high productivity) while having access to human judgment for complex situations (high adaptability).
Solution Approach 2:
The system prepares for potential autonomous failures by establishing a remote takeover capability in advance. The VR remote control system and communication infrastructure are pre-configured so that when the autonomous vehicle detects it cannot handle a situation, control can be seamlessly transferred to a remote operator. This preliminary preparation ensures continuous operational capability without requiring permanent human presence in the vehicle.
Data Source
AI summary
In various examples, at least partial control of a vehicle may be transferred to a control system remote from the vehicle. Sensor data may be received from a sensor(s) of the vehicle and the sensor data may be encoded to generate encoded sensor data. The encoded sensor data may be transmitted to the control system for display on a virtual reality headset of the control system. Control data may be received by the vehicle and from the control system that may be representative of a control input(s) from the control system, and actuation by an actuation component(s) of the vehicle may be caused based on the control input.