Prediction Consensus Outlier Filtering Against Divergent Model Outputs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer models generating event predictions often diverge significantly and may not align with observable data, leading to inconsistent and potentially biased consensus values.
Innovation Solution
A prediction consensus system that evaluates predicted event values from multiple models, identifies outliers using statistical measures, and generates consensus values incorporating both predicted and observed data, while maintaining model training integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If consensus values are generated from multiple model predictions, then the reliability of predictions is improved, but the accuracy deteriorates due to significant divergence among model predictions and observed data
Solution Approach 1:
The patent extracts and removes outlier predictions from the consensus calculation process. By identifying predictions that fall outside the interquartile range (IQR) of the distribution and removing them before calculating the consensus value, the system eliminates harmful divergent predictions while maintaining the benefits of multi-model consensus for reliable predictions
Solution Approach 2:
The patent applies different treatment to different predictions based on their statistical properties. Rather than treating all predictions uniformly, the system identifies specific predictions as outliers based on their deviation from the distribution and applies selective removal, thereby improving accuracy locally for problematic predictions while preserving accurate predictions
2Productivity
If all predicted event values are used to generate consensus values, then the productivity of the prediction system is improved, but the manufacturing precision deteriorates due to inclusion of outlier predictions
Solution Approach 1:
The patent applies partial action by using only a subset of predictions (those within the IQR range) rather than all predictions. This selective approach removes excessive outlier predictions that would degrade precision while maintaining efficiency by still processing multiple models and using automated statistical methods to identify and remove only the necessary outliers
3Measurement precision
If statistical measures are used to identify outliers, then the measurement precision of prediction evaluation is improved, but the device complexity increases due to additional processing requirements
Solution Approach 1:
The patent changes the parameter of outlier detection from complex machine learning-based methods to simple statistical parameters (mean, standard deviation, and interquartile range). This transformation maintains high measurement precision in identifying outliers while significantly reducing device complexity by using straightforward mathematical calculations that can be implemented with minimal computational resources
Data Source
AI summary
A prediction consensus system receives predicted event values for an event. The predicted event values are generated by various computer models that may have different training schemes and input data. The prediction consensus system generates a consensus value based on the predicted event values after excluding outliers. Outliers are determined among the predicted event values based on statistical measures of the predicted event values as well as observed event values. The predicted event values that appear to be outliers based on the predicted statistical measures may nonetheless be included when they are within the observed statistical measures. This ensures the predicted event values used in the consensus values are not unduly biased towards other predicted event values. Similarly, the identification of event values as accepted or an outlier may be returned as training data for retraining the computer models that generated the predicted event values.