Decoder-Only Sequence Classification Explanations via Intermediate Predictions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating input attributions in decoder-only sequence models are either expensive or yield low-quality explanations that do not accurately reflect the model's behavior.
Innovation Solution
A method and system for computing input attributions using intermediate predictions by evaluating decoder-only sequence models at different points in the input sequence, employing techniques like Kernel SHapley Additive exPlanations (SHAP) and progressive inference to generate high-quality explanations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If previous methods (input perturbations, relevance propagation, attention scores, or gradients) are used to generate input attributions, then the process is simpler to implement, but the quality of attributions is low and does not accurately reflect model behavior
Solution Approach 1:
The patent introduces intermediate predictions as a mediator between the original input and final model prediction. By evaluating the model at multiple intermediate points during the decoding process and aggregating these predictions, the method obtains more accurate attributions that reflect the model's actual reasoning trajectory, rather than relying on post-hoc approximation methods
Solution Approach 2:
The patent segments the model's prediction process into multiple intermediate steps during the decoding phase. Instead of treating the model as a black box that maps input to output, the method divides the transformation into discrete intermediate predictions that can be individually analyzed and aggregated to reveal the model's decision-making process
2Reliability
If accurate input attributions are obtained through intermediate predictions, then model trustworthiness and explanation quality improve, but computational cost increases
Solution Approach 1:
The patent performs the computationally intensive intermediate prediction evaluations during the model's forward pass as a preliminary action, rather than requiring separate expensive perturbation analyses afterward. By integrating attribution computation into the existing decoding process, the method obtains reliable explanations without multiplying the computational cost
3Measurement precision
If intermediate predictions are used to compute attributions, then attribution accuracy improves, but the time required for explanation generation increases
Solution Approach 1:
The patent merges the attribution computation process with the model's existing inference pipeline by integrating intermediate prediction collection into the decoding loop. This combination allows the system to generate accurate attributions without requiring separate explanation generation steps, thereby reducing the time overhead compared to traditional post-hoc explanation methods
Data Source
AI summary
Methods and systems for computing input attributions to accurately explain predictions of decoder-only sequence classification models are provided. The method includes: receiving a set of inputs to the decoder-only sequence classification model; generating, based on the first set of inputs, a perturbed version of the set of inputs; sampling a binary mask from a predetermined masking distribution; generating a group of masked versions of the perturbed set of inputs by applying the binary mask to the perturbed set of inputs; generating, based on the group of masked versions of the perturbed set of inputs, corresponding sets of intermediate predictions that correspond to the decoder-only sequence classification model; computing, based on the sets of intermediate predictions, a set of input attributions; and determining, based on the set of input attributions, an explanation that relates to a prediction of the decoder-only sequence classification model.