Neural Network Training Data Correction via Influence Approximation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current techniques for improving machine learning model predictive accuracy are laborious and time-consuming, particularly in identifying and correcting erroneous training data, as they often require re-annotating entire datasets and do not attribute causal influence to annotations within training samples at a sub-sample level.
Innovation Solution
The system identifies and corrects erroneous training data at an intra-sample and sub-sample level by computing the influence of each training label on incorrect predictions using conditional loss approximation, allowing for targeted re-annotation and improving predictive accuracy through a user interface for correcting labels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional techniques are used to improve machine learning model predictive accuracy by re-annotating data and iterating over hyperparameters, then predictive accuracy may be improved, but the process becomes laborious and time-consuming
Solution Approach 1:
The patent segments the training data analysis at the sub-sample level, examining individual training examples and their contributions to specific predictions. This segmentation allows the system to identify only the specific erroneous training examples causing incorrect predictions, rather than requiring comprehensive re-annotation of entire datasets, thus reducing time loss while maintaining accuracy improvement.
Solution Approach 2:
The patent introduces an intermediary mechanism (influence function analysis) that mediates between the training data and model predictions. This intermediary approach enables automated identification of problematic training examples through mathematical analysis of model behavior, replacing laborious manual re-annotation processes while preserving the ability to improve predictive accuracy.
2Reliability
If comprehensive re-annotation of training data is performed to correct errors, then predictive accuracy may be improved, but the complexity and labor required increases significantly
Solution Approach 1:
The patent extracts and isolates only the specific problematic training examples that are causing incorrect predictions, rather than requiring comprehensive re-annotation of entire datasets. This extraction approach reduces the scope of correction work to minimal necessary elements, decreasing process complexity while maintaining accuracy improvement benefits.
Solution Approach 2:
The system performs self-service by automatically identifying erroneous training examples through influence function analysis without requiring manual inspection of all training data. The model analyzes its own training data contributions to incorrect predictions, reducing the complexity of human-in-the-loop correction processes while achieving accurate error identification.
3Difficulty of detecting and measuring
If traditional heuristic methods are used to diagnose errors in training data, then some errors may be identified, but the method lacks precision in attributing causal influence to specific annotations
Solution Approach 1:
The patent implements a feedback mechanism where the model's predictions are analyzed to provide information about which training examples are causing errors. This feedback loop enables precise attribution of causal influence by measuring the impact of removing or modifying specific training examples on prediction outcomes, significantly improving measurement precision compared to traditional heuristic methods.
Solution Approach 2:
The patent changes the analytical parameters from simple error detection to influence measurement by computing how model predictions change when training examples are removed or modified. This parameter transformation enables precise quantification of causal influence, allowing the system to identify not just that errors exist, but exactly which training annotations are causing specific incorrect predictions.
Data Source
AI summary
One aspect of systems and methods for data correction includes identifying a false label from among predicted labels corresponding to different parts of an input sample, wherein the predicted labels are generated by a neural network trained based on a training set comprising training samples and training labels corresponding to parts of the training samples; computing an influence of each of the training labels on the false label by approximating a change in a conditional loss for the neural network corresponding to each of the training labels; identifying a part of a training sample of the training samples and a corresponding source label from among the training labels based on the computed influence; and modifying the training set based on the identified part of the training sample and the corresponding source label to obtain a corrected training set.