Supervised Machine Learning Validation With Corrected Failure Probability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current supervised machine learning methods fail to provide sufficient evidence of safety for AI algorithms used in safety-critical applications, making it challenging to optimize safety-relevant processes during operation.
Innovation Solution
A method for supervised machine learning that involves creating a data pool with input and output data sets, dividing them into training and validation sets, training the algorithm using the training sets, and validating it using the validation sets, with a statistical significance correction factor to ensure a lower corrected probability of failure than the target probability, thereby ensuring safety and reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If supervised machine learning is used to optimize safety-relevant applications, then productivity and efficiency are improved, but reliability and safety certification become problematic due to insufficient evidence of safety
Solution Approach 1:
The patent applies preliminary action by performing validation on training data before the machine learning model is deployed for safety-critical operations. The validation process includes checking data quality,代表性 (representativeness), and completeness of the training dataset beforehand, ensuring that the model will meet safety requirements when deployed. This advance validation provides the necessary evidence for safety certification before actual operational use.
Solution Approach 2:
The patent implements feedback mechanisms by continuously monitoring the machine learning model's performance against safety criteria during validation. The system provides feedback on whether the training data and model outputs meet predetermined safety standards, allowing for iterative improvement and adjustment of the training process to ensure safety certification requirements are met.
2Speed
If machine learning algorithms are trained with available data, then learning speed is improved, but measurement precision and validation accuracy deteriorate due to statistical significance issues
Solution Approach 1:
The patent applies parameter changes by adjusting the validation threshold and statistical significance criteria based on the specific safety requirements of the application. The system dynamically modifies validation parameters such as confidence intervals, significance levels, and acceptance criteria to balance learning speed with validation accuracy, ensuring that faster training does not compromise the precision of safety validation.
3Reliability
If the validation threshold is set to ensure high safety standards, then reliability is improved, but productivity decreases due to stricter validation requirements
Solution Approach 1:
The patent applies segmentation by dividing the validation process into multiple independent stages: data quality validation, model performance validation, and safety criterion validation. Each stage has specific thresholds and criteria that can be adjusted independently. This segmentation allows the system to maintain high safety standards while improving overall validation efficiency by focusing computational resources on critical validation points rather than applying uniform strict thresholds across all validation aspects.
Data Source
AI summary
A supervised machine learning of a computer-implemented method for performing a technical process in which a data pool is created. The data pool contains data sets with input and output data that describes a correct process result. The data sets are divided into training and validation data sets. The computer-implemented method is trained in a training phase, wherein process parameters of the method are varied during repeated performances. The trained method is checked by comparing the output data calculated with the trained method with the input data with the output data describing the correct process result and calculating an actual probability of failure based on the comparison. An empirical probability of failure is defined for the training phase, which is at most as large as a target probability of failure, the validation phase is initiated after it has been established that the empirical probability of failure is not exceeded.