Probabilistic Confidence Labels for Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine-learning models for classification, such as artificial neural networks, fail to effectively capture inter-class similarity information during training, leading to poor performance due to confusion between similar classes, as existing labeling techniques do not explicitly account for these similarities.
Innovation Solution
The method involves labeling objects with probabilistic confidence labels that include weighted probability classification scores based on multiple labelers' or models' outputs, using a projective loss function to relax penalties for errors that confuse similar classes, and normalizing these labels to ensure they sum to a constant value, allowing the model to capture and exploit inter-class similarities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional classification labels are used for training, then the training process is simple, but the model performance deteriorates due to confusion between similar classes
Solution Approach 1:
The patent transforms traditional discrete classification labels into continuous probabilistic confidence labels. Each label becomes a probability distribution across multiple classes, capturing inter-class similarity information. This parameter transformation allows the model to distinguish between similar classes by learning from the nuanced probability relationships rather than binary classifications.
Solution Approach 2:
The patent introduces probabilistic confidence labels as an intermediary between traditional labels and model predictions. These intermediate labels encode similarity information and guide the model to understand relationships between classes, acting as a bridge that transfers similarity knowledge from labeled data to the learning process.
2Reliability
If probabilistic confidence labels with inter-class similarity information are incorporated, then model performance improves, but the labeling process becomes more complex
Solution Approach 1:
The patent performs preliminary computation of probabilistic confidence labels during the data preparation phase. By pre-calculating probability distributions that encode inter-class similarities and storing them as training labels, the complex labeling work is done upfront rather than during model inference, simplifying the overall system architecture.
Solution Approach 2:
The patent enables the training data itself to provide similarity information through the structure of probabilistic labels. The labels self-encode inter-class relationships through their probability distributions, eliminating the need for separate similarity computation mechanisms during model training.
3Reliability
If standard loss functions are used for training, then the training process is straightforward, but the model cannot effectively learn from noisy labels with inter-class similarities
Solution Approach 1:
The patent modifies the loss function to operate on probability distributions rather than discrete labels. By changing the mathematical parameters of the loss computation to handle continuous probability vectors, the model can naturally accommodate noisy labels and inter-class similarities without requiring complex preprocessing or postprocessing steps.
Data Source
AI summary
Provided is a system, method, and computer program product for training a machine-learning model. The method includes labeling each object of a plurality of objects with a probabilistic confidence label including a probability classification score for each class of at least two classes, resulting in a plurality of probabilistic confidence labels associated with the plurality of objects, and training, with at least one computing device, the machine-learning model based on the plurality of objects and the plurality of probabilistic confidence labels.