Machine Learning Validation Model for Prediction Reliability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The accuracy of prediction data generated by supervised learning models in machine learning systems for mechanical systems is uncertain, making it difficult for users to determine the reliability of the output and whether to retrain the models.
Innovation Solution
A machine learning apparatus incorporating a prediction model and a validation model, where the validation model generates accuracy data based on input data, allowing users to assess the reliability of the prediction model's output through various indices and graphs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a supervised learning model is used to generate prediction data, then the model can provide structured output, but the accuracy and reliability of the prediction data becomes uncertain
Solution Approach 1:
The patent introduces an unsupervised learning model as an intermediary component that processes the same input data independently. This intermediary model generates accuracy data that serves as a reliability indicator for the supervised learning model's predictions, allowing users to assess confidence without requiring complex explanations or additional verification processes
Solution Approach 2:
The system implements a feedback mechanism where the unsupervised learning model continuously monitors and evaluates the output of the supervised learning model. The accuracy data generated by the unsupervised model provides real-time feedback on prediction reliability, enabling dynamic adjustment of trust levels in the prediction data based on actual performance metrics
2Reliability
If users want to determine the reliability of prediction data, then they need to assess accuracy, but this requires complex evaluation processes and additional computational resources
Solution Approach 1:
The unsupervised learning model performs self-evaluation by processing the same input data independently and generating accuracy data that automatically reflects the reliability of the supervised model's predictions. This self-service mechanism eliminates the need for external verification processes, manual auditing, or complex evaluation frameworks, simplifying the reliability assessment while maintaining accuracy
Solution Approach 2:
The system creates a copy of the evaluation function through the unsupervised learning model, which replicates the input data processing pathway. This copy operates independently to generate parallel output that can be compared with the supervised model's predictions, providing a simplified copying-based verification mechanism that reduces computational overhead compared to exhaustive validation approaches
3Adaptability or versatility
If users need to decide whether to retrain models based on prediction accuracy, then they require reliable accuracy information, but this creates uncertainty about when retraining is necessary
Solution Approach 1:
The unsupervised learning model continuously generates accuracy data in advance, providing predictive indicators before actual performance degradation occurs. This preliminary action allows users to proactively decide when retraining is needed based on trending accuracy metrics rather than reacting to failed predictions, improving adaptability while reducing uncertainty about timing
Data Source
AI summary
A machine learning apparatus includes a model holder, a data storage, and a model computing unit. The model holder holds a first machine learning model having undergone supervised learning and a second machine learning model having undergone unsupervised learning. The model computing unit inputs input data to the first machine learning model to generate first output data and inputs the input data also to the second machine learning model to generate accuracy data. The accuracy data is calculated based on a value in at least one of the input layers, the middle layer, and the output layer of the second machine learning model such that the accuracy data changes its tendency in response to the first output data.