Continuous Output Model Evaluation Using Discretized Accuracy Bins
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine-learning algorithms struggle to accurately predict events impacting machine-implemented operating environments, necessitating improved evaluation and updating methods for continuous output variables.
Innovation Solution
A system and method for evaluating modeling algorithms using continuous variables, generating a discretized evaluation dataset, and computing an evaluation metric to assess accuracy, enabling selection and updating of higher-performing models to adapt machine-implemented environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine-learning algorithms are used to predict continuous output variables, then predictive capability is improved, but evaluation accuracy remains insufficient
Solution Approach 1:
The patent segments the continuous evaluation metric into discrete categories by binning continuous values into discrete bins. This transforms the continuous evaluation output into a categorical format that can be reliably evaluated using standard classification metrics, thereby resolving the issue of insufficient evaluation accuracy for continuous output variables.
Solution Approach 2:
The patent changes the parameter representation from continuous to discrete by applying binning to the continuous evaluation metric. This parameter transformation enables the use of reliable evaluation metrics designed for categorical data,ไป่ improving evaluation accuracy while maintaining predictive capability.
2Adaptability or versatility
If continuous output variables are used for prediction, then model flexibility is improved, but evaluation difficulty increases
Solution Approach 1:
The patent segments the continuous evaluation metric into discrete categories through binning. This segmentation transforms the difficult-to-evaluate continuous metric into a categorical format that is straightforward to measure using standard evaluation metrics, thereby reducing evaluation difficulty while preserving model flexibility.
Solution Approach 2:
The patent introduces a discrete binning layer as an intermediary between the continuous model output and the evaluation metric. This intermediary transformation simplifies the measurement and evaluation process while maintaining the underlying continuous predictive capability of the model.
3Device complexity
If existing evaluation metrics are used for continuous variables, then computational simplicity is maintained, but evaluation reliability deteriorates
Solution Approach 1:
The patent segments the continuous evaluation metric into discrete categories, enabling the use of reliable categorical evaluation metrics. This segmentation improves evaluation reliability while maintaining computational simplicity by using standard binning and classification evaluation techniques.
Solution Approach 2:
The patent changes the parameter representation from continuous to discrete through binning, allowing the use of reliable evaluation metrics for categorical data. This parameter change improves evaluation reliability without significantly increasing computational complexity.
Data Source
AI summary
Certain aspects involve evaluating modeling algorithms whose outputs can impact machine-implemented operating environments. For instance, a computing system generates, from a comparison of a set of estimated attribute values of an attribute to a set of validation attribute values of the attribute, a discretized evaluation dataset with data values in multiple categories. The computing system computes, for a modeling algorithm used to generate the estimated attribute values, an evaluation metric. The computing system provides a host computing system with access to the evaluation metric, one or more modeling outputs generated with the modeling algorithm, or both. Providing one or more of these outputs to the host computing system can facilitate modifying one or more machine-implemented operations.