IT Ticket Classification With Abstention and Explainable Labels

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current IT ticketing systems face challenges in accurately classifying technical issues and preventing erroneous automated changes to software and hardware components, as general language classifiers struggle with technical data and lack explainability.

Innovation Solution

A client-agnostic machine learning model is trained to classify IT domain-related tickets while abstaining from non-relevant ones, using a linear classifier algorithm and abstention mechanism to prevent misclassification, and provides explainable decisions using disjunctive normal form and pertinent positive features.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If general language classifiers are used to classify IT tickets, then the system can handle general text, but it fails to accurately classify technical issues and provides no explanation for decisions

Engineering Contradiction:

Improveability to handle general textVSAvoidclassification accuracy for technical issues

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent applies local quality by training separate specialized classifiers for different domains (IT technical issues, general text, finance, healthcare) rather than using a single general classifier. Each classifier is optimized for its specific domain, achieving high accuracy for technical issues while maintaining the ability to handle various text types through domain-specific expertise

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent introduces an intermediary classification layer that first determines the domain of the ticket before routing to specialized classifiers. This intermediary step enables the system to handle general text efficiently while ensuring technical issues are processed by domain-expert classifiers, resolving the contradiction between versatility and precision

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If automated classification systems process all tickets, then productivity increases, but erroneous automated changes occur when tickets are misclassified

Engineering Contradiction:

Improveticket processing speedVSAvoidaccuracy of automated changes

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent applies preliminary action by implementing an abstention mechanism that evaluates confidence levels before automated classification. Tickets that fall outside the model's expertise or have low confidence scores are flagged for manual review before automated processing, preventing erroneous changes while maintaining high productivity for confident classifications

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent implements feedback loops where classification results are continuously evaluated and used to improve the model. The abstention mechanism provides feedback on uncertain cases, allowing the system to learn from edge cases and improve reliability over time while maintaining automated processing for clear-cut cases

Inventive Principle:

Principle #23Feedback

3Extent of automation

If machine learning models classify all incoming tickets, then automation extent increases, but the models cannot identify when they should abstain from classification

Engineering Contradiction:

Improveautomated classification coverageVSAvoidability to recognize domain boundaries

Core Design Contradiction:

Extent of automationVSAdaptability or versatility

Solution Approach 1:

The patent applies dynamics by making the classification system adaptive through the abstention mechanism. The model dynamically adjusts its behavior based on input characteristics, automatically determining when to classify and when to abstain. This dynamic approach allows high automation coverage for suitable tickets while maintaining the ability to recognize and defer uncertain cases

Inventive Principle:

Principle #15Dynamics

4Device complexity

If general classifiers are used without domain-specific training, then device complexity is reduced, but classification precision for technical domains deteriorates

Engineering Contradiction:

Improvemodel architecture simplicityVSAvoidtechnical issue classification accuracy

Core Design Contradiction:

Device complexityVSMeasurement precision

Solution Approach 1:

The patent applies segmentation by dividing the classification system into multiple specialized models, each trained on specific domains (IT, finance, healthcare). This segmentation maintains relative simplicity of individual models while achieving high precision for technical domains through domain-specific training data and features

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS20240345905A1Explainable classifications with abstention using client agnostic machine learning models

Publication Date: 2024.10.17 KYNDRYL INC

AI summary

Embodiments relate to providing explainable classifications with abstention using client agnostic machine learning models. A technique includes inputting, by a processor, records to a machine learning model, the records being associated with an information technology (IT) domain. The technique includes classifying, by the processor, the records with labels using the machine learning model, the machine learning model abstaining from classifying a given record in response to the given record being outside of a scope of the IT domain.