ML Output Correction Using Acceptance Tests and Safe-Region Adjustment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI systems face challenges in maintaining reliable and robust performance over time due to domain drifts, leading to suboptimal and unstable model outputs that can result in safety hazards and decreased operational efficiency.
Innovation Solution
A computer-implemented method for correcting model outputs of a first trained machine learning model using a second trained machine learning model, which applies correction parameters based on acceptance test verdicts to ensure the model outputs remain within a safe region.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the high-performance controller is retrained to adapt to domain drift, then model output accuracy is improved, but the complexity and time consumption of the system increases
Solution Approach 1:
The system is segmented into two specialized components: a high-performance controller for normal operation and a correction model for handling failure cases. This segmentation allows each component to be optimized for its specific function - the high-performance controller maintains high accuracy during normal operation while the correction model handles adaptation to domain drift, avoiding the need for continuous retraining of the entire system and reducing overall complexity.
Solution Approach 2:
The correction model acts as an intermediary between the high-performance controller and the safe region constraints. Instead of directly retraining the complex high-performance controller to adapt to domain drift, the correction model intercepts unsafe outputs and transforms them into safe outputs, serving as a mediator that handles adaptation without requiring fundamental changes to the primary controller.
2Reliability
If the high-assurance controller operates for longer periods to ensure safety, then safety is improved, but energy consumption increases and throughput decreases
Solution Approach 1:
Instead of continuously operating the conservative high-assurance controller, the system applies partial correction only when and where needed - specifically when the high-performance controller produces unsafe outputs. The correction model applies minimal necessary corrections to push outputs back into the safe region, avoiding excessive use of the high-assurance controller and maintaining high throughput while ensuring safety.
Data Source
AI summary
A computer-implemented method for correcting at least one model output of a first trained machine learning model is provided, including providing the at least one model output of the first trained machine learning model; verifying the at least one model output using at least one acceptance test resulting in at least one test verdict regarding a fail or a pass of the at least one model output; determining at least one correction parameter using a second trained machine learning model based on the at least one test verdict and the at least one model output, if the first trained machine learning model fails the at least one acceptance test; d. adjusting the at least one model output of the first trained machine learning model by applying the at least one correction parameter; and providing the at least one adjusted output as at least one corrected model output.