Machine Learning Output Augmentation With Training Support Metrics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional machine learning models often produce unreliable outputs due to inadequate training data, leading to high confidence scores without sufficient training basis, and consumers lack means to differentiate between reliable and unreliable outputs.
Innovation Solution
Integrate training-support-based augmentation by analyzing training data distributions and generating metrics to augment model outputs, providing confidence scores alongside training support metrics, allowing consumers to adjust or discard outputs based on training adequacy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional machine learning models are trained on available data, then the model can produce classification or regression outputs, but the model cannot indicate the reliability of its outputs when training data is not similar to input data
Solution Approach 1:
The patent segments the model's output into two distinct components: the primary prediction output and a separate training support metric. This segmentation allows the model to provide both the classification/regression result and an independent reliability indicator without conflating the two functions, thereby resolving the contradiction between producing outputs and indicating their reliability.
Solution Approach 2:
The patent introduces a training support metric as an intermediary element that bridges the gap between the model's internal training characteristics and the external output reliability assessment. This intermediary metric quantifies the similarity between training data and input data, providing a measurable indicator of output reliability without requiring fundamental changes to the model's core prediction function.
2Reliability
If the model provides only classification or regression outputs, then the output structure is simple, but consumers cannot distinguish between reliable and unreliable determinations
Solution Approach 1:
The patent applies local quality by adding the training support metric specifically to augment the output in cases where reliability indication is needed, rather than fundamentally restructuring the entire output system. The metric is computed and attached locally to each prediction, providing reliability information only where necessary while maintaining overall output simplicity.
Solution Approach 2:
The patent adds a new dimension to the output by introducing the training support metric as an additional parameter. This transforms the output from a single-value prediction to a multi-dimensional result that includes both the prediction and its reliability indicator, allowing consumers to assess determination quality without complicating the core prediction function.
3Reliability
If the model is trained without considering data distribution characteristics, then the training process is efficient, but the model cannot identify when training data is insufficient for specific inputs
Solution Approach 1:
The patent implements preliminary action by computing the training support metric during the training phase, establishing a baseline understanding of data distribution characteristics before the model is deployed. This preliminary computation enables the model to later assess whether new input data falls within the distribution patterns observed during training, providing reliability assessment capability without requiring complex real-time analysis.
Solution Approach 2:
The patent introduces feedback by using the training support metric to inform both the training process and the prediction process. The metric provides feedback on data distribution alignment, allowing the model to identify when training data is insufficient for specific inputs and adjust accordingly, thereby improving reliability assessment while maintaining training efficiency through targeted rather than comprehensive retraining.
Data Source
AI summary
Machine learning models are provided that consider, during the process of producing output, various aspects of the training data and/or training process from which the models are created. A machine learning model may generate output (e.g., classification determinations or regression output) that is augmented with information regarding the distribution(s) of the corpus of training data upon which the model was trained, the features extracted from the training data, the resulting determinations made by the model, and/or other information. The augmentation may occur internally while generating the model output, or the output itself may be augmented to include distribution-based data in addition to a model output.