Centroidal Classifier for Text Interpretation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

End users lack understanding and trust in the operation of computer-implemented text classifiers due to their 'black box' nature, and existing solutions to provide explanatory outputs are costly and lack robustness as they require rewriting or developing specific classifiers.

Innovation Solution

A separate centroidal classifier is trained to mimic the operation of the text classifier, generating class profiles that represent its operation, allowing users to understand classification decisions through similar texts and highlighted terms, without modifying the original classifier.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If a deep neural network is employed in connection with assigning a classification to input text, then accuracy of the text classifier is improved, but understanding and trust of the classifier by end users deteriorates due to its black box nature

Engineering Contradiction:

Improveclassification accuracyVSAvoidunderstandability of classification decisions

Core Design Contradiction:

Measurement precisionVSLoss of information

Solution Approach 1:

The patent introduces an intermediary explanation module that sits between the black box classifier and the end user. This module translates the classifier's internal reasoning into human-understandable explanations by identifying and presenting the most influential features and their contributions to the classification decision, thereby maintaining classification accuracy while improving user understanding

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent creates a simplified copy or representation of the classifier's decision-making process through feature importance analysis. Instead of exposing the complex neural network internals, it generates a simplified model that copies the essential reasoning logic in terms of influential features, making the classification decisions transparent without sacrificing the original classifier's accuracy

Inventive Principle:

Principle #26Copying

2Loss of information

If code is added to a text classifier to generate explanatory output describing its operations, then user understanding is improved, but device complexity and development cost increase due to the need to rewrite or redevelop the classifier

Engineering Contradiction:

Improveexplanatory output qualityVSAvoidclassifier structure complexity

Core Design Contradiction:

Loss of informationVSDevice complexity

Solution Approach 1:

The patent segments the classifier system into two independent components: the original classification module and a separate explanation generation module. The explanation module independently analyzes the classifier's decisions by examining feature importances and generating explanations without requiring modifications to the original classifier code, thus avoiding increased device complexity

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent creates a universal explanation framework that can work with any text classifier architecture. The explanation module uses general-purpose techniques like feature importance analysis and SHAP values that are architecture-agnostic, allowing it to provide explanations for different classifier types without requiring classifier-specific code, thereby reducing overall system complexity

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Loss of information

If code is developed for a particular text classifier to generate explanatory output, then that specific classifier gains interpretability, but robustness deteriorates as the code is not applicable to other classifiers

Engineering Contradiction:

Improvemodel interpretabilityVSAvoidapplicability across different classifier architectures

Core Design Contradiction:

Loss of informationVSAdaptability or versatility

Solution Approach 1:

The patent develops a universal explanation framework based on model-agnostic techniques such as SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations). These techniques can be applied to any classifier architecture by treating the classifier as a black box and analyzing its input-output relationships, making the explanation code robust and applicable across different classifiers without requiring architecture-specific modifications

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent introduces an intermediary explanation layer that mediates between the classifier and the user. This layer uses universal explanation techniques that do not depend on the internal architecture of the classifier, allowing it to provide consistent interpretability across different classifier types while maintaining the original classifiers' integrity and performance

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS11880660B2Interpreting text classifier results with affiliation and exemplification

Publication Date: 2024.01.23 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

Technologies relating to model interpretation are described herein. A text classifier is provided with input text and assigns a class to the input text from amongst several possible classes. Based upon the class assigned to the input text by the text classifier, a class profile of a centroidal classifier is selected, where the class profile is constructed based upon numerous input texts to which the text classifier has previously assigned the class. Based upon the selection of the class profile, information that is indicative of operation of the text classifier with respect to the text classifier assigning the class to the input text is output, where the information includes an exemplar text sequence.