Conflict Detection for Modular Intent Classifiers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cognitive question answering (QA) systems face instability and unpredictability when combining independently trained intent classifiers due to overlapping intent definitions, leading to conflicts and unpredictable outcomes, as there are no effective methods for evaluating and resolving these overlaps automatically.
Innovation Solution
A system and method for detecting and resolving conflicts between independently trained intent classifiers by using conflict detection and resolution engines, which evaluate overlapping intent definitions, provide alerts, and offer automated recommendations for revising training data or redefining intents to prevent unstable decisions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple independently trained intent classifiers are combined to handle different application domains, then the system's versatility and coverage are improved, but conflicts and instability arise due to overlapping intent definitions
Solution Approach 1:
The patent applies preliminary action by performing conflict detection and resolution analysis before deploying multiple intent classifiers into production. The system evaluates potential overlaps between classifiers using test data and confusion matrices, identifying conflicting intent pairs in advance. This allows developers to resolve conflicts through retraining or intent redefinition before the classifiers are combined, preventing instability from occurring during actual operation rather than reacting to it afterward.
2Measurement precision
If manual trial-and-error processes are used to evaluate and resolve classifier conflicts, then customization and precision are improved, but time consumption and operational complexity increase significantly
Solution Approach 1:
The patent implements self-service by enabling the system to automatically evaluate intent classifier conflicts without requiring manual intervention. The conflict detection mechanism automatically feeds test data through multiple classifiers, collects their predictions, builds confusion matrices, and identifies conflicting intent pairs autonomously. Resolution recommendations are generated automatically based on the detected conflicts, eliminating the need for developers to manually test each classifier combination and significantly reducing the time required for conflict resolution while maintaining high precision through systematic evaluation.
3Measurement precision
If confidence score calibration is improved to make scores more reliable, then the accuracy of individual classifier predictions is improved, but conflicts between classifiers with overlapping intents remain unstable and unpredictable
Solution Approach 1:
The patent introduces an intermediary conflict detection and resolution layer between the individual intent classifiers and the final decision-making process. This intermediary mechanism uses confusion matrices to identify which intent pairs are likely to conflict when classifiers compete. By detecting these conflicts in advance and providing resolution recommendations (such as retraining classifiers or redefining intents), the system prevents unstable competition from occurring, allowing confidence scores to remain reliable indicators of individual classifier performance without being undermined by inter-classifier conflicts.
Data Source
AI summary
A method, apparatus, and system are provided for resolving conflicts between training data conflicts by retrieving independent training data sets, each comprising a plurality of intents and end-user utterances for use in training one or more classifiers to recognize a corresponding intent from one or more of the end-user utterances, providing a first test end-user utterance associated with a first intent from the first independent training data set to the one or more classifiers to select an output intent generated by the one or more classifiers; identifying a first conflict when the first intent does not match the output intent, and automatically generating, by the system, one or more conflict resolution recommendations for display and selection by an end user to resolve the first conflict.


