Remote Pilot Sensor Compositing by Area of Focus
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Full autonomous control of vehicles faces challenges in certain contexts, and remote control of vehicles is proposed to maintain human control while ensuring efficient operation, but the efficiency of data transmission between the remotely controlled vehicle and the remote pilot terminal is crucial for safe operation.
Innovation Solution
A system that processes sensor data from multiple sensors with overlapping fields of view to create a composite data stream, prioritizing data based on the remote pilot's area of focus, reducing network bandwidth and latency, and allowing for efficient communication and rendering of the vehicle's environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If all sensor data feeds are transmitted with equal quality, then complete environmental information is provided to the remote pilot, but network bandwidth consumption increases and latency increases
Solution Approach 1:
The patent applies local quality by differentiating the quality of sensor data transmission based on spatial location. The compositor prioritizes sensor data from the area of focus (where the remote pilot is looking) with high quality transmission, while reducing or omitting transmission of data from peripheral areas. This resolves the contradiction by providing complete environmental information where needed (area of focus) while conserving network bandwidth in less critical regions.
Solution Approach 2:
The system dynamically adjusts data transmission quality based on the remote pilot's area of focus, which changes over time. The compositor continuously monitors which sensors are in the area of focus and prioritizes their data transmission accordingly. This dynamic adaptation allows the system to maintain information completeness in relevant areas while optimizing bandwidth usage as the pilot's attention shifts.
2Reliability
If multiple sensors with overlapping fields of view are used, then comprehensive environmental monitoring is achieved, but data processing complexity increases
Solution Approach 1:
The compositor extracts and prioritizes only the necessary sensor data from the multiple sensor feeds based on the area of focus. Instead of processing all sensor data equally, the system extracts relevant data from sensors whose fields of view intersect with the area of focus, thereby reducing processing complexity while maintaining comprehensive monitoring coverage through the coordinated operation of multiple sensors.
3Measurement precision
If high resolution sensor data is transmitted from all sensors, then detailed environmental representation is achieved, but transmission time increases
Solution Approach 1:
The patent applies local quality by transmitting high-resolution sensor data only from sensors located in the area of focus, while transmitting lower resolution or compressed data from peripheral sensors. This resolves the contradiction by providing detailed environmental representation where the pilot needs it most (in the area of focus) while reducing overall transmission time through selective quality adjustment.
Data Source
AI summary
Processing sensor data from a plurality of sensors monitoring an exterior environment of a remotely controlled vehicle. The sensors generate a corresponding respective plurality of sensor data feeds. A compositor for combining the plurality of sensor data feeds into a composite data stream is provided. A remote pilot terminal is operative to receive the composite data stream and render a representation of the exterior environment of the remotely controlled vehicle to a remote pilot. A pilot monitor determines an area of focus of the remote pilot, and the area of focus is provided to the compositor for use in differentiating the plurality of data feeds in the composite data stream. The representation of the exterior environment at the remote operator terminal is based on the area of focus of the remote pilot.