Machine Learning Model Validation and Retraining via Error Image Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users face a heavy burden in determining which images to use for training machine learning models, leading to trial and error, increased training time, and potential over-training, as well as difficulty in confirming classification accuracy in existing image processing devices.
Innovation Solution
An image processing device with a processor that automatically trains, validates, and re-trains a machine learning model by inputting images, evaluating classification accuracy, and selecting validation images with high error possibilities for re-training until sufficient accuracy is achieved, reducing user burden and optimizing training efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If the user manually determines which images to use for training, then the training process can be controlled, but the user burden increases and trial and error occurs
Solution Approach 1:
The system automatically determines which validation images should be used for re-training by evaluating classification accuracy and identifying images with high error possibilities, eliminating the need for user intervention in the training selection process
Solution Approach 2:
The system evaluates classification accuracy using validation images and feeds back this information to automatically determine which images should be used for re-training, creating a closed-loop system that continuously improves without user burden
2Measurement precision
If the user confirms classification accuracy manually, then the accuracy can be verified, but the user burden increases
Solution Approach 1:
The system automatically evaluates classification accuracy by inputting validation images and calculating the degree of separation between classes, eliminating the need for manual user confirmation
Solution Approach 2:
The manual confirmation process is replaced with automated computational evaluation using the machine learning model to calculate classification accuracy and degree of separation between classes
3Reliability
If additional training is performed when accuracy is insufficient, then the model performance improves, but the training process becomes complex and time-consuming
Solution Approach 1:
Instead of complete re-training with all training images, the system selectively uses only those validation images that have high error possibilities for re-training, reducing the training workload while maintaining performance improvement
Solution Approach 2:
The system preliminarily evaluates classification accuracy and identifies specific images with high error possibilities before initiating re-training, allowing targeted and efficient training without unnecessary complexity
Data Source
AI summary
A processor: executes classification of classifying a plurality of validation images into a plurality of classes with a machine learning model trained with a plurality of training images; obtains a degree of separation between the plurality of classes by the classification of the plurality of validation images and evaluates accuracy of the classification of the plurality of validation images based on the obtained degree of separation between the plurality of classes; and evaluates whether re-training of the machine learning model is necessary based on an evaluation result of the accuracy of classification of the plurality of validation images, extracts an validation image whose classification result has a relatively high possibility to be erroneous from among the plurality of validation images to automatically re-train the machine learning model if it is evaluated that the re-training of the machine learning model is necessary.