Autonomous Vehicle Display Overlay for Remote Feature Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles often struggle to provide a comprehensive and clear environmental overview to human operators, leading to potential misinterpretation and suboptimal decision-making during remote control scenarios.
Innovation Solution
Combining video frame data with perception output data to generate a 2D or 3D rendered graphical model, allowing operators to view both perspectives simultaneously and enabling user inputs to enhance or correct the model, thereby providing a more informed understanding of the vehicle's environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If only video frame data is displayed to the operator, then the display is simple and easy to understand, but the operator lacks comprehensive environmental information leading to potential misinterpretation
Solution Approach 1:
The patent combines video frame data with perception output data into a unified display interface. The video data provides real-time visual context while the perception data overlays processed environmental information, creating a comprehensive view that merges raw sensor data with interpreted environmental models without requiring separate displays.
Solution Approach 2:
The patent adds a new dimension to the display by overlaying perception data (such as detected objects, paths, and environmental features) onto the video frame data. This creates a multi-layered display where operators can view both the raw video feed and the processed environmental interpretation simultaneously, enhancing information density without proportionally increasing complexity.
2Reliability
If multiple data sources are integrated to provide comprehensive environmental overview, then operator understanding is enhanced, but the system complexity increases
Solution Approach 1:
The patent segments the display into distinct functional areas: video frame data display area, perception output data display area, and user input interface. This segmentation allows each data source to be processed and displayed separately while maintaining overall system integration, reducing the complexity burden of handling multiple data sources simultaneously.
Solution Approach 2:
The patent introduces an intermediary processing layer that receives both video frame data and perception output data, processes them through coordinate system transformations and spatial alignments, and presents them in a unified display format. This intermediary layer manages the complexity of data integration by providing standardized interfaces between different data sources and the final display output.
3Loss of information
If video frame data and perception output data are displayed separately, then each data source is clear and distinct, but the operator cannot easily correlate features between the two views
Solution Approach 1:
The patent uses color coding to differentiate between video frame data elements and perception output data elements in the unified display. Detected objects, paths, and environmental features are highlighted with distinct colors or visual markers that maintain clarity while enabling easy correlation between the two data sources within the same visual field.
Data Source
AI summary
A method is provided, comprising: (i) receiving, at a system, from an autonomous vehicle: (a) sensor data captured by a sensor of the vehicle, and (b) output data generated based on environmental data captured by one or more sensors of the vehicle and used by a planning component to navigate in an environment, (ii) causing one or more displays to display at least one of: a representation of the sensor data, or a model of the environment based on the output data, (iii) determining a location of a feature within the sensor data or the model, (iv) causing the one or more displays to display an indication of the feature at a position corresponding to the location, (v) receiving, at the system, user input, and (vi) sending, by the system to the vehicle, data based on the user input to cause the vehicle to take an action.