Attention Layer Explainability for Clinical NLP Assertion Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for classifying assertion statuses in clinical natural language sentences are insufficiently interpretable or explainable, making it difficult to audit or trust the outputted classification labels.
Innovation Solution
A system comprising a deep learning neural network with a hidden attention layer that generates assertion status classification labels and extracts word-wise attention scores, which are then rendered to provide explainability and interpretability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a deep learning neural network is used to classify assertion statuses, then classification accuracy is improved, but interpretability and explainability deteriorate
Solution Approach 1:
The patent introduces attention scores as an intermediary between the deep learning neural network and the classification output. These attention scores serve as a mediator that reveals which words in the input sentence most influenced the classification decision, thereby providing interpretability without compromising the accuracy of the underlying neural network model.
Solution Approach 2:
The patent segments the classification process into distinct components: the deep learning neural network for accurate classification, and an attention mechanism that breaks down the decision-making process by assigning scores to individual words. This segmentation allows the system to maintain high accuracy while providing transparent explanations through word-level attention scores.
2Loss of information
If attention scores are extracted and visualized to improve explainability, then system complexity increases
Solution Approach 1:
The attention mechanism serves multiple functions simultaneously: it improves the accuracy of the neural network by focusing on relevant words, provides interpretability through attention scores, and enables visualization of the decision-making process. This multi-functionality reduces the need for separate explanation systems, thereby limiting the increase in overall system complexity.
3Loss of information
If word-wise attention scores are extracted from hidden layers, then processing time increases
Solution Approach 1:
The attention scores are computed as part of the forward propagation process through the neural network, before the final classification decision is made. This preliminary computation of attention scores during the natural processing flow avoids the need for separate post-processing steps, thereby minimizing the additional processing time required for extraction and visualization.
Data Source
AI summary
Systems or techniques that facilitate systems and methods for providing explainability of natural language processing are provided. In various embodiments, a system can access a plain text clinical sentence. In various aspects, the system can generate, via execution of a first machine learning model, an assertion status classification label for a word of interest in the plain text clinical sentence. In various instances, the system can extract, from a hidden attention layer of the first machine learning model, word-wise attention scores corresponding to the plain text clinical sentence and render, on an electronic display, both the assertion status classification label and a graphical representation of the word-wise attention scores. In various cases, the system can determine, via execution of a second machine learning model, a reliability score for the assertion status classification label, based on the word-wise attention scores, and can render the reliability score on the electronic display.