Model Update Support System for Classification Anomaly Diagnosis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models used for data classification face challenges in accurately updating their training data, leading to anomalies where data is misclassified due to insufficient or inappropriate training, making it difficult for users to determine the cause of the issue.
Innovation Solution
A model update support system that includes a processor to calculate classification certainty and similarity between data and labeled data, providing users with information on whether the model's training is insufficient or if a label is inappropriate, allowing for targeted retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the model is trained using deep learning for data classification, then the classification capability is improved, but the difficulty in determining the cause of misclassification increases
Solution Approach 1:
The patent segments the anomaly detection process into distinct components: calculating classification certainty for the input data, calculating similarities between input data and training data, and using these segmented metrics to determine the cause. This segmentation allows the system to isolate whether the issue lies in model confidence or data representation, making the diagnosis process systematic and manageable.
Solution Approach 2:
The patent introduces intermediary metrics (classification certainty and similarity values) that mediate between the complex deep learning model and the user. These intermediaries translate the model's internal state into interpretable measures, allowing users to understand whether low classification accuracy stems from insufficient training or inappropriate labels without needing to comprehend the complex model internals.
2Measurement precision
If the model training data is updated to improve classification accuracy, then the model performance is improved, but the time and resources required for retraining increase
Solution Approach 1:
The patent performs preliminary analysis by calculating classification certainty and similarity metrics before actual retraining occurs. This preliminary action identifies which specific training samples are problematic (low similarity) or which classifications are uncertain, allowing users to selectively retrain only on necessary data rather than retraining the entire model, thus reducing time and resource consumption.
Solution Approach 2:
The patent enables partial retraining by identifying specific problematic training samples through similarity comparison. Instead of retraining with all training data (excessive action), the system allows users to focus retraining efforts only on the subset of data that shows low similarity to new inputs or has low classification certainty, making the retraining process more efficient and targeted.
3Measurement precision
If the similarity between input data and training data is calculated for all training samples, then the accuracy of anomaly detection is improved, but the computational complexity increases
Solution Approach 1:
The patent extracts only the necessary similarity information needed for anomaly detection rather than computing all possible metrics. By focusing specifically on similarity between input data and training samples in the context of classification certainty, the system obtains sufficient diagnostic information without the computational burden of comprehensive analysis, achieving a balance between accuracy and complexity.
Data Source
AI summary
According to one embodiment, a model update support system supports an update of a first model trained using a training data group. The training data group includes a plurality of labeled data and includes a plurality of labels respectively labeling the plurality of labeled data. The system includes a processor. The processor is configured to output first information or second information based on a classification certainty and a plurality of similarities. The classification certainty is calculated using the first model and indicates a sureness of a classification of first data. The plurality of similarities respectively indicates likenesses between the first data and the plurality of labeled data. The first information indicates that the training of the first model is insufficient. The second information indicates that one of the plurality of labels is inappropriate.