Remote Vehicle Teleoperation With Gaze-Based Video Prioritization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Semi-autonomous vehicles face challenges in teleoperated driving due to fluctuations in mobile network connection quality, leading to interruptions in video transmission and impaired controllability, affecting both vehicle occupants and road users.
Innovation Solution
The method involves predictive adaptation of sensor data transmission by prioritizing and preprocessing data based on connection quality, viewing direction, and data flow requirements, using redundant connections to ensure stable teleoperated driving even with fluctuating network conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sensor data is transmitted at high data rates to ensure quality video transmission for teleoperated driving, then the viewing quality for the teleoperator is improved, but interruptions occur during poor connection quality, impairing controllability
Solution Approach 1:
The system dynamically adapts the data transmission rate and quality based on real-time connection quality predictions. When connection quality is good, high-resolution video data is transmitted at high rates. When connection quality degrades, the system automatically reduces data rates and adjusts quality parameters to maintain continuous transmission, preventing interruptions and maintaining controllability throughout varying network conditions
Solution Approach 2:
The system changes transmission parameters including data rate, resolution, and quality settings based on predicted connection quality. By adjusting these parameters dynamically, the system ensures that video transmission maintains adequate quality for teleoperation while adapting to fluctuating network conditions, thus preventing interruptions and maintaining reliable control
2Loss of information
If all sensor data is transmitted with high quality to ensure comprehensive information for the teleoperator, then the information completeness is improved, but the bandwidth consumption increases, leading to interruptions under fluctuating connection quality
Solution Approach 1:
The system applies different quality levels to different portions of sensor data based on their importance and the teleoperator's viewing direction. Critical areas and high-priority sensor data maintain high quality transmission, while less critical data uses reduced quality settings. This selective quality approach preserves essential information while significantly reducing overall bandwidth consumption, preventing interruptions under fluctuating connection conditions
3Quantity of substance
If the data transmission rate is increased to provide detailed sensor data for complex driving scenarios, then the data detail level is improved, but the system becomes more vulnerable to connection quality fluctuations and interruptions
Solution Approach 1:
The system dynamically adjusts the volume and detail level of sensor data transmission based on real-time connection quality predictions. During periods of good connection quality, the system transmits comprehensive high-detail sensor data. When connection quality deteriorates, the system automatically reduces data volume and detail level, ensuring continuous stable transmission without interruptions while still providing sufficient data for safe teleoperation
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method (30) for controlling a vehicle (10) by means of a remote operator, characterized by the following features: - while the remote operator is looking at an image based on sensor data (18) of the vehicle (10), a line of vision (19) of the remote operator is captured, - data streams (20, 21, 22) that the sensor data (18) comprise are assigned different priorities on the basis of the line of vision (19) (2), - a specified function is used to make a prediction (24) in regard to a quality of the data connection, - the priorities and the quality of the data connection are used to adapt specifications (25) in regard to a respective data rate of the data streams (20, 21, 22), and - the specifications (25) are taken as a basis for transmitting the sensor data (18) from the vehicle (10) to the control station (17) for the purpose of updating the image.