Driving Scenario Visualization for Interpreting Rule Evaluation Failures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Interpreting the complex rule evaluations in autonomous vehicle testing is challenging due to the nuanced and multi-conditioned nature of driving scenarios, making it difficult for users to understand the direct relationship between rule evaluations and scenario events.
Innovation Solution
A graphical user interface is developed to visualize driving scenarios with time plots of numerical performance scores, allowing users to select time markers to see how rule failures correspond to actual events, enhancing interpretability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If rule-based models are used to evaluate autonomous vehicle performance, then comprehensive safety standards can be established, but the complexity of interpreting test results increases significantly
Solution Approach 1:
The patent introduces an intermediary visualization system that mediates between the complex rule-based evaluation results and the user. This system includes a scenario visualization component that displays the driving scenario context, a rule evaluation component that processes rule results, and a visualization component that presents simplified visual indicators (pass/fail markers, timelines) to bridge the gap between complex data and user comprehension.
Solution Approach 2:
The patent segments the complex evaluation results into distinct visual components: individual rule evaluations are separated and displayed as discrete elements on a timeline, with each rule showing its specific pass/fail status. The scenario is also segmented into time-based segments that can be independently analyzed, allowing users to focus on specific events rather than overwhelming comprehensive data.
2Reliability
If multiple rules are defined to cover various driving scenarios, then evaluation comprehensiveness improves, but user interpretability of individual rule failures deteriorates
Solution Approach 1:
The patent adds a temporal dimension to the visualization by arranging rule evaluations along a timeline that corresponds to the scenario progression. This allows users to understand not only which rules failed but when they failed relative to specific scenario events. The visualization component displays multiple dimensions simultaneously: scenario context, time progression, and rule evaluation status, creating a multi-dimensional view that enhances interpretability.
Solution Approach 2:
The patent creates visual copies or representations of the scenario events alongside the rule evaluations. The scenario visualization component generates a visual representation of the driving scenario that can be directly compared with the rule evaluation results, allowing users to see the direct correspondence between scenario events and rule failures without having to mentally map abstract rule descriptions to concrete events.
3Measurement precision
If detailed numerical scores are provided for each rule, then measurement precision improves, but ease of understanding by users decreases
Solution Approach 1:
The patent employs color changes as a visual encoding mechanism to represent rule evaluation outcomes. Different colors indicate different statuses (e.g., green for pass, red for fail, yellow for warning), allowing users to quickly comprehend rule results at a glance. This color-coded system preserves the precision of numerical scoring while translating it into an intuitively understandable visual format that can be processed rapidly by users.
Data Source
AI summary
A computer system for rendering a graphical user interface for visualising runs of a driving scenario in which an ego agent navigates a road layout, comprising an input configured to receive a map of the road layout and run data comprising a sequence of timestamped ego agent states and a time-varying numerical score quantifying the performance of the ego agent with respect a set of run evaluation rules; and a rendering component configured to cause a graphical user interface to display, for each rule: a plot of the time-varying numerical score, and a marker denoting a selected time index of the plot, the marker movable along the time axis to change the selected time index; and a scenario visualization comprising a visualization of the run at the selected time index, whereby moving the marker along the time axis causes the scenario visualisation to update as the time index is changed.