Neural Network Layer Confidence Calibration for Chatbot Overprediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks in chatbot systems suffer from overconfidence issues, where the confidence scores generated for class predictions become de-correlated from the real confidence scores, leading to inaccurate and difficult-to-post-process model outputs.
Innovation Solution
A method is provided to generate a distribution of confidence scores for each layer of a machine learning model, determine a prediction based on these scores, iteratively process layers to identify a layer satisfying a criterion, and assign an overall confidence score to the model's prediction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep neural networks are used to improve prediction accuracy, then the accuracy of output predictions is improved, but the confidence scores become de-correlated from real confidence scores causing overconfidence
Solution Approach 1:
The patent segments the neural network into multiple layers and introduces separate prediction modules at different layers (intermediate layers and output layer). Each segment generates its own confidence scores, allowing the system to identify the most reliable prediction by comparing segments rather than relying on a single overconfident output.
Solution Approach 2:
The patent implements a feedback mechanism where confidence scores from intermediate layers are used to evaluate and select the final prediction. The system feeds back the confidence score distribution from earlier layers to the prediction selection process, using this feedback to identify predictions with reliable confidence scores rather than simply taking the output from the final layer.
2Measurement precision
If deep neural networks with many layers are used, then prediction accuracy is improved, but the complexity of the model increases
Solution Approach 1:
The patent extracts the prediction and confidence score generation functionality from just the output layer and places identical prediction modules at multiple intermediate layers as well. This extraction allows the system to use a simpler, repeated module structure rather than complex interactions between deep layers, reducing model complexity while maintaining accuracy.
Solution Approach 2:
The patent adds a new dimension to the neural network architecture by introducing multiple prediction modules across different layers rather than relying solely on depth. This dimensional change from vertical depth-dependence to horizontal layer-distribution allows the system to achieve accuracy through ensemble selection rather than through increased model complexity.
3Reliability
If confidence scores are generated for each layer, then the overconfidence problem is addressed, but the computational processing time increases
Solution Approach 1:
The patent applies partial action by not using all layer outputs equally but instead selecting only the prediction with the most reliable confidence score. The system performs confidence score generation at multiple layers (excessive action) but then applies selection criteria to use only the necessary information (partial action), reducing processing time while maintaining reliability.
Solution Approach 2:
The patent performs preliminary confidence score generation at intermediate layers before the final output is produced. By having confidence scores ready from earlier layers, the system can quickly evaluate prediction reliability without waiting for complete forward propagation through all layers, reducing overall processing time.
Data Source
AI summary
Disclosed herein are techniques for addressing an overconfidence problem associated with machine learning models in chatbot systems. For each layer of a plurality of layers of a machine learning model, a distribution of confidence scores is generated for a plurality of predictions with respect to an input utterance. A prediction is determined for each layer of the machine learning model based on the distribution of confidence scores generated for the layer. Based on the predictions, an overall prediction of the machine learning model is determined. A subset of the plurality of layers are iteratively processed to identify a layer whose assigned prediction satisfies a criterion. A confidence score associated with the assigned prediction of the layer of the machine learning model is assigned as an overall confidence score to be associated with the overall prediction of the machine learning model.