Explainable ML Classifications with Domain Abstention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current IT ticketing systems face challenges in accurately analyzing and categorizing technical issues, as general language classifiers fail to handle technical data effectively and do not provide explanations for their decisions, leading to misclassifications and incorrect automated actions.
Innovation Solution
A client-agnostic machine learning model is developed to classify IT tickets within its scope, abstaining from misclassifying tickets outside the IT domain and providing explainable decisions using disjunctive normal form and pertinent positive features, ensuring accurate and trustworthy automated resolution processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If general language classifiers are used to categorize IT tickets, then the system can handle general text, but it fails to accurately handle technical data and provides no explanation for decisions
Solution Approach 1:
The patent segments the classification task into two distinct components: a domain detection module that first determines whether the ticket belongs to the IT domain, and a classification module that only processes tickets confirmed to be within the IT domain. This segmentation prevents general language classifiers from misclassifying non-technical tickets while maintaining their ability to handle general text.
Solution Approach 2:
The patent introduces an intermediary domain detection module between the input ticket and the classification system. This intermediary acts as a gatekeeper that filters out non-IT domain tickets before they reach the classifier, thereby improving classification accuracy for technical data without sacrificing the system's ability to handle general text inputs.
2Productivity
If automated classification is implemented, then productivity increases, but reliability decreases due to misclassifications and incorrect automated actions
Solution Approach 1:
The patent performs preliminary domain detection before classification to ensure that only appropriate tickets are classified automatically. This preliminary action prevents misclassifications of non-technical tickets, thereby maintaining reliability while preserving productivity through automated processing of genuine IT tickets.
Solution Approach 2:
The system incorporates feedback mechanisms where classification results are evaluated against domain appropriateness. When misclassifications are detected or when domain uncertainty exists, the system can adjust its behavior, providing a feedback loop that continuously improves reliability without significantly impacting productivity.
3Productivity
If the machine learning model classifies all incoming records, then complete coverage is achieved, but misclassifications of non-IT domain records increase
Solution Approach 1:
The patent implements a dynamic classification system that adapts its behavior based on the detected domain of each ticket. The system dynamically adjusts between classification mode (for IT domain tickets) and abstention mode (for non-IT domain tickets), optimizing both coverage and accuracy by making real-time decisions about whether to classify each record.
4Reliability
If explainable AI is implemented, then user confidence increases, but system complexity increases
Solution Approach 1:
The patent segments the explainability function into a separate explanation generation module that works alongside the classification system. This module independently generates human-readable explanations for classification decisions without fundamentally altering the core classification architecture, thereby increasing user confidence while minimizing added complexity.
Data Source
AI summary
Embodiments relate to providing explainable classifications with abstention using client agnostic machine learning models. A technique includes classifying, by a processor, a record with a label using a machine learning model, the machine learning model abstaining from classifying a given record in response to the given record being outside of a scope of an information technology (IT) domain. The processor generates an explanation of a decision by the machine learning model to classify the record with the label and displays the explanation in a human readable form.