AI Clinical Decision Support Trust Calibration Through User Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing clinical decision support systems face challenges in achieving appropriate user trust levels, with users either over-trusting or under-trusting system outputs due to inaccuracies, leading to suboptimal decision-making and potential errors in healthcare delivery.
Innovation Solution
A system that evaluates user interactions with AI-based CDS systems to detect over-trust or under-trust by monitoring user behaviors and generating corrective actions, such as visual, auditory, or tactile prompts, to align user trust with system accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the CDS system provides AI-based outputs to assist clinical decision-making, then decision-making efficiency is improved, but user trust accuracy deteriorates due to over-trust or under-trust
Solution Approach 1:
The system implements a feedback mechanism that monitors user interactions with CDS outputs and provides corrective feedback to adjust user trust levels. The system evaluates user trust based on interaction patterns and provides targeted feedback to encourage appropriate trust calibration, resolving the contradiction by continuously adapting user trust based on system performance and user behavior.
Solution Approach 2:
The system performs preliminary evaluation of user trust levels before critical decisions are made. By assessing user trust accuracy in advance through interaction monitoring and providing corrective actions proactively, the system prevents both over-trust and under-trust from compromising decision-making reliability.
2Reliability
If the system monitors user interactions to evaluate trust levels, then user trust accuracy is improved, but system complexity increases
Solution Approach 1:
The system uses passive monitoring of naturally occurring user interactions to evaluate trust levels, requiring no additional user effort or specialized monitoring equipment. The system self-services by extracting trust evaluation data from routine user behavior patterns, thereby improving trust accuracy without proportionally increasing system complexity.
3Reliability
If corrective actions are provided to adjust user trust, then user trust accuracy is improved, but ease of operation deteriorates due to additional prompts and notifications
Solution Approach 1:
The system applies corrective actions locally and selectively rather than universally. Corrective prompts and notifications are provided only when specific trust calibration needs are detected based on individual user interaction patterns, rather than applying uniform corrections to all users. This targeted approach improves trust accuracy while minimizing disruptions to ease of operation.
Data Source
Figure 1
Figure 2
Figure 3A~3D
AI summary
Techniques for evaluating user interactions with a clinical decision support (CDS) system are disclosed. User interaction data is received associated with a user response to an advice item generated by the CDS system. The user interaction data is evaluated, such as to determine a user trust level associated with the advice item and/or the CDS system. A corrective action is generated based on the evaluation of the user interaction data. In some examples, the disclosed techniques identify and correct for user under-trust or over-trust in artificial intelligence (AI)-based CDS systems.