Probabilistic Loss Function for Machine Learning Uncertainty
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional machine learning systems lack the ability to differentiate between outputs based on their accuracy probability, leading to inefficiencies in training and deployment, requiring significant time and resources to distinguish between accurate and inaccurate outputs, and resulting in longer development periods, increased costs, and lower performance.
Innovation Solution
The introduction of a novel loss function that incorporates an uncertainty probability (UPS) and an adjustable penalty value, allowing the system to segregate outputs based on confidence levels, where inputs with high uncertainty are flagged for human expert review and those with low uncertainty are generated as outputs, optimizing resource allocation and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional ML systems are used without uncertainty differentiation, then the system structure remains simple, but the accuracy of outputs varies significantly and cannot be differentiated
Solution Approach 1:
The patent segments outputs into two distinct categories: certain outputs (from inputs similar to training data) and uncertain outputs (from inputs dissimilar to training data). This segmentation enables the system to differentiate accuracy levels without requiring complex post-processing, as the uncertainty probability is inherently generated during the forward propagation phase.
Solution Approach 2:
The patent adds a new dimension to the output by introducing an uncertainty probability value alongside the conventional output. This additional dimension (uncertainty probability) allows the system to convey not just the prediction result but also the confidence level, enabling accuracy differentiation without fundamentally restructuring the existing ML architecture.
2Measurement precision
If post-training methods are applied to distinguish accurate from inaccurate outputs, then some differentiation capability is achieved, but development time increases considerably
Solution Approach 1:
The patent implements uncertainty differentiation during the training phase by modifying the loss function to include an uncertainty component. This preliminary action embeds the differentiation capability directly into the model training process, eliminating the need for time-consuming post-training analysis and threshold selection by data scientists.
Solution Approach 2:
The modified loss function enables the ML system to automatically differentiate between certain and uncertain outputs without requiring external intervention from data scientists. The system self-services by inherently generating uncertainty probabilities and using them to categorize outputs, eliminating manual threshold tuning and post-processing steps.
3Productivity
If conventional ML systems process all inputs uniformly, then the system operation is simple, but resources are wasted on inputs that will produce inaccurate outputs
Solution Approach 1:
The patent applies different handling strategies to different types of inputs based on their uncertainty probability. Certain inputs (high similarity to training data) are processed and output automatically, while uncertain inputs (low similarity) are flagged for human review. This local quality approach optimizes resource allocation by applying intensive resources only where necessary rather than uniformly to all inputs.
Solution Approach 2:
The patent applies partial action by not processing all inputs through the complete ML pipeline. Instead, it uses uncertainty assessment to determine which inputs require full processing and which can be quickly identified and routed differently, reducing overall resource consumption while maintaining accuracy for critical cases.
Data Source
AI summary
A probabilistic loss function based machine learning system (MLS) is disclosed. The MLS can generate an additional new class for uncertainty value(s) determined for inputs to the MLS. The uncertainty value(s) can be correlated to output(s) of the MLS. A novel loss function can be based on a conventional loss function, the uncertainty value(s), and an adjustable penalty value. The adjustable penalty value can be adjusted, such that improving the optimization of the novel loss function can cause the MLS to reduce use of computing resources in determining outputs corresponding to uncertainty value(s) above a threshold value(s) in favor of using computing resource for determining outputs corresponding to uncertainty value(s) below the threshold value(s). Updating of the adjustable penalty value can result in changes to the threshold value(s).