Chatbot Logit Scaling for Out-of-Domain Intent Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional chatbot systems face challenges in accurately classifying out-of-domain or out-of-scope utterances due to overconfidence issues in deep neural networks, leading to inaccurate classification of user intents.
Innovation Solution
Implementing enhanced logit values in machine-learning models to predict whether an utterance corresponds to a resolvable or unresolvable class, using statistical, bounded, weighted, hyperparameter-optimized, or learned values to improve classification accuracy for out-of-scope utterances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional deep neural networks are used for classifying utterances, then the system can process user inputs efficiently, but the classification accuracy deteriorates for out-of-domain or out-of-scope utterances due to overconfidence issues
Solution Approach 1:
The patent modifies the logit function by introducing a temperature parameter that scales the logits before applying the softmax function. This temperature parameter allows the system to adjust the confidence distribution, reducing overconfidence for out-of-domain utterances while maintaining efficiency. The modified logit function transforms the original logits into adjusted logits that better reflect true uncertainty.
2Adaptability or versatility
If the chatbot system uses standard logit functions, then the model structure remains simple, but the ability to handle ambiguous and out-of-domain user inputs deteriorates
Solution Approach 1:
The patent segments the classification process into multiple stages: first computing standard logits through the neural network, then applying a separate temperature-based scaling transformation, and finally computing the softmax distribution. This segmentation allows the system to handle out-of-domain utterances more effectively without fundamentally changing the core model architecture, maintaining relative simplicity while improving adaptability.
3Measurement precision
If the system increases confidence scores for all classifications, then the user experience improves through more decisive responses, but the accuracy deteriorates for ambiguous or unresolvable classes
Solution Approach 1:
The temperature parameter serves as a control mechanism that dynamically adjusts confidence levels. For resolvable classes, the system maintains appropriate confidence, while for ambiguous or out-of-domain utterances, the temperature scaling reduces excessive confidence. This allows the system to provide decisive responses when appropriate while accurately recognizing uncertainty in ambiguous cases.
Data Source
AI summary
Techniques for using enhanced logit values for classifying utterances and messages input to chatbot systems in natural language processing. A method can include a chatbot system receiving an utterance generated by a user interacting with the chatbot system and inputting the utterance into a machine-learning model including a series of network layers. A final network layer of the series of network layers can include a logit function. The machine-learning model can map a first probability for a resolvable class to a first logit value using the logit function. The machine-learning model can map a second probability for a unresolvable class to an enhanced logit value. The method can also include the chatbot system classifying the utterance as the resolvable class or the unresolvable class based on the first logit value and the enhanced logit value.