Explainable AI System for Autonomous Vehicle Trust Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Human users of autonomous vehicles often experience anxiety or discomfort when observing unexpected maneuvers, leading to reduced trust and potential avoidance of using the vehicle in the future, as existing systems lack effective communication to mitigate these feelings.
Innovation Solution
A method and system that generate and present tailored explanations for the vehicle's actions to users, using a model trained on trust levels from test subjects, optimizing the selection of explanations to maximize user trust through a probabilistic approach and mutual information measures, balancing explanation complexity and information content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the autonomous vehicle performs unexpected maneuvers to optimize traffic navigation, then the vehicle's productivity and navigation efficiency are improved, but the human passenger's trust and comfort level deteriorate
Solution Approach 1:
The system implements feedback by presenting explanations for autonomous actions to passengers and measuring their trust responses. This feedback loop allows the system to learn which explanations effectively increase trust and adjust future explanation strategies accordingly, resolving the contradiction between performing optimized maneuvers and maintaining user trust.
Solution Approach 2:
The explanation system acts as an intermediary between the autonomous vehicle's decision-making process and the passenger's perception. By introducing explanations as a mediating element, the system translates unexpected maneuvers into understandable rationale, thereby maintaining trust while allowing productivity-optimizing actions to occur.
2Reliability
If the system provides detailed explanations for each maneuver, then user trust increases, but the device complexity and information processing requirements worsen
Solution Approach 1:
The system changes parameters by selecting from different explanation types (e.g., goal-oriented, risk-based, plan-based) and adjusting explanation detail levels based on measured passenger responses. This allows the system to optimize trust-building while managing complexity through parameter adjustment rather than structural complexity.
Solution Approach 2:
The system applies partial action by providing explanations selectively rather than for every single action. The explanation frequency and detail are adjusted based on what is sufficient to maintain trust, avoiding the excessive complexity that would result from explaining every maneuver in full detail.
3Measurement precision
If the system collects and processes trust level data from multiple test subjects to build a comprehensive model, then the measurement precision of trust prediction improves, but the loss of time and data processing requirements worsen
Solution Approach 1:
The system performs preliminary action by collecting and processing trust level data from test subjects in advance during offline model training. This allows the computationally intensive data processing to occur before deployment, so that during actual operation the system can quickly select explanations based on the pre-built model without time-consuming real-time analysis.
Data Source
AI summary
An autonomous vehicle and a system a method of operating a machine. The system includes a processor. A set of explanations related to a machine behavior of the machine is generated. The processor generates a model that relates an explanation for the behavior taken in response to a scenario to a trust level that a human has in the behavior when the explanation is presented to the human, the explanation being selected from the set of explanations. The processor performs the behavior of the system of vehicle in response to the scenario, uses the model to select the explanation when the behavior is taken, and presents the explanation to the human.