Just in Time Classifier Training for Missing Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning classifiers face challenges when dealing with missing values in data features, as they often require training on complete datasets, and existing methods for handling missing values can impact the validity of classification results or require impractical calibration for each possible missing feature combination.
Innovation Solution
A method that identifies key features in a test dataset and modifies the training dataset to only include those present in the test dataset, allowing for just-in-time calibration of the classifier, which assesses the importance of each feature by perturbing it with negative samples and measuring the change in classifier output, enabling efficient processing of datasets with missing values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the classifier is trained on complete datasets with all features, then the training accuracy is improved, but the classifier cannot handle real-world data with missing values
Solution Approach 1:
The system dynamically adapts the training process by identifying which features are actually present in the test data and recalibrating the classifier accordingly. Instead of using a fixed training approach, the system adjusts the training features based on the specific missing value patterns encountered in each test dataset, allowing the classifier to handle various missing value scenarios effectively
Solution Approach 2:
The system changes the parameters of the training process by selectively including or excluding features based on their presence in the test data. The calibration process modifies the training dataset to match the actual features available in the test data, transforming the classifier's behavior to accommodate different data completeness scenarios
2Quantity of substance
If missing values are handled by inserting placeholder values, then the data completeness is improved, but the classification accuracy deteriorates due to meaningless inserted values
Solution Approach 1:
The system extracts and identifies the actual features that are present in the test data, separating them from the complete feature set used during initial training. By focusing only on the features that actually exist in the test data, the system avoids using meaningless placeholder values and recalibrates the classifier based on relevant features only
Solution Approach 2:
Instead of trying to fill in missing values to make the data complete, the system inverts the approach by identifying which values are actually present and using only those for calibration. This reverses the conventional wisdom of completing data and instead focuses on utilizing available data effectively
3Adaptability or versatility
If the classifier is recalibrated for every possible combination of missing features, then the handling of missing values is improved, but the computational complexity becomes impractical
Solution Approach 1:
The system performs partial calibration by focusing only on the features that are actually present in the test data rather than recalibrating for all possible feature combinations. This partial action approach provides sufficient adaptability for the specific test case without the excessive computational burden of complete recalibration for all scenarios
Solution Approach 2:
The system performs self-calibration by automatically identifying which features are present in the test data and adjusting its own training parameters accordingly. This self-service mechanism eliminates the need for external intervention or pre-computation of all possible calibration scenarios, reducing complexity while maintaining adaptability
Data Source
AI summary
Disclosed herein is a system and method that can be used with any underlying classification technique. The method receives a test dataset and determines the features in that test dataset that are present. From these features the training dataset is modified to only have those features that are present in the test dataset. This modified test dataset is then used to calibrate the classifier for the particular incoming data set. The process repeats itself for each different incoming dataset providing a just in time calibration of the classifier.