Uncertainty Quantification With Automated Pareto Model Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current uncertainty quantification systems require user judgment for selecting evaluation methods, leading to inefficiencies and potential biases in model simulations and technical systems.
Innovation Solution
An automated uncertainty quantification system that builds surrogate models using a dual objective Pareto optimal model selection process, incorporating Bayesian optimization and active learning to enhance model selection and reduce uncertainties, while providing user interfaces for data refinement and model performance metrics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated uncertainty quantification system is implemented, then run-time efficiency and accuracy are improved, but device complexity increases
Solution Approach 1:
The system performs automated model selection and uncertainty quantification without requiring user expertise. The automated application independently executes the dual objective Pareto optimal model selection process, selecting appropriate surrogate models based on the data characteristics and evaluation criteria, thereby eliminating the need for user judgment while maintaining high efficiency and accuracy
Solution Approach 2:
The patent introduces a dual objective Pareto optimal model selection process as an intermediary layer between raw data and final uncertainty quantification results. This intermediary process automatically evaluates multiple surrogate models against dual objectives (accuracy and uncertainty calibration) and selects the optimal model, resolving the complexity issue by structuring the automated process into manageable, systematic steps
2Adaptability or versatility
If user judgment is required for selecting evaluation methods, then model selection flexibility is improved, but reliability decreases due to potential biases
Solution Approach 1:
The system incorporates feedback mechanisms where the automated application evaluates surrogate models based on dual objectives (accuracy and uncertainty calibration) and uses this feedback to iteratively select and refine model choices. This feedback-driven approach ensures consistent, bias-free selection while maintaining adaptability to different data types and problem domains
Solution Approach 2:
The patent changes the selection criteria from user-defined subjective parameters to objective, data-driven parameters including accuracy metrics and uncertainty calibration measures. The dual objective Pareto optimal selection process automatically adjusts model parameters based on performance feedback, ensuring reliable and consistent model selection across different applications without requiring user judgment
3Measurement precision
If dual objective Pareto optimal model selection process is used, then measurement precision is improved, but loss of time increases due to comprehensive evaluation
Solution Approach 1:
The automated application implements partial evaluation by assessing surrogate models against dual objectives but stopping the evaluation process once sufficient discrimination between model quality levels is achieved. Rather than exhaustively evaluating all possible models to maximum precision, the system performs enough evaluation to make reliable model selection, thereby reducing time loss while maintaining adequate measurement precision
Solution Approach 2:
The system performs preliminary filtering of surrogate models based on basic criteria before applying the full dual objective Pareto optimal evaluation. This preliminary action eliminates obviously inferior models early in the process, reducing the number of models requiring comprehensive evaluation and thereby decreasing time loss while preserving measurement precision for the final model selection
Data Source
AI summary
According to an embodiment, a computer-implemented method for operating a system for uncertainty quantification (UQ) of imperial data, a simulation of a mathematical model or for testing a technical system includes the following steps: (i) defining simulation output parameters and an accuracy range; (ii) uploading simulation output data and configuration file; (iii) searching for and applying a variety of surrogate models via automation; (iv) determining the reliability of the selected model; (v) providing a report; and (vi) using the validated model to generate new data points.