Impact-Scored Data Selection for Neural Network Retraining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training neural network models is computationally expensive and time-consuming, and using inappropriate external data can deteriorate performance and increase training time, necessitating a system to minimize training time while maintaining high accuracy.
Innovation Solution
An electronic device determines impact scores for training and external data points, selectively updating training data based on these scores to prioritize effective data points for re-training, thereby reducing the number of data points and training time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the neural network model is trained using all available external data points, then the model accuracy is improved, but the training time and computational cost increase significantly
Solution Approach 1:
The patent segments the external data into multiple groups based on their impact scores. Instead of treating all data points uniformly, the system divides data into high-impact and low-impact segments, selectively using only the high-impact segments for re-training. This segmentation allows the model to maintain accuracy while reducing training time by focusing only on the most beneficial data portions.
Solution Approach 2:
The patent applies local quality by differentiating the treatment of different data points based on their individual impact scores. High-impact data points are selected for re-training while low-impact points are excluded. This localized approach ensures that training resources are concentrated on data that provides the most value for improving model accuracy, thereby reducing overall training time.
2Reliability
If the neural network model is re-trained with inappropriate external data, then the model performance deteriorates, but the training time increases
Solution Approach 1:
The patent performs preliminary action by calculating impact scores for all external data points before re-training begins. This pre-evaluation step identifies which data points are likely to improve model performance and which may deteriorate it. By selecting only high-impact data points for re-training, the system prevents performance deterioration while minimizing training time.
Solution Approach 2:
The patent implements feedback through the impact score calculation, which evaluates the potential effect of each external data point on model performance before inclusion in the training set. This feedback mechanism allows the system to adaptively select data points that are most likely to improve performance, avoiding the inclusion of inappropriate data that would increase training time without benefiting the model.
3Loss of time
If the number of data points used for training is reduced, then the training time is minimized, but the model accuracy may be compromised
Solution Approach 1:
The patent changes the parameter of data selection from uniform inclusion to impact-score-based selection. By using impact scores as a filtering parameter, the system reduces the number of data points used for training while maintaining accuracy. The impact score threshold acts as a parameter that controls the balance between training time and accuracy, allowing optimization based on specific requirements.
Data Source
AI summary
An electronic device includes a memory to store neural network model trained for classification tasks of real-time applications. The neural network model is trained with plurality of training data points. The electronic device includes circuitry to retrieve a plurality of external data points. The electronic device applies the neural network model on the plurality of external data points to determine a plurality of impact scores for each external data point. The plurality of impact scores indicates amount of contribution of each training data point towards a prediction of each external data point. The electronic device selects a set of external data points based on the plurality of impact scores. The electronic device updates the plurality of training data points with the set of external data points to generate a second plurality of training data points and re-trains the neural network model based on the second plurality of training data points.