ML Training Dataset Balancing via Sample Contribution Ranking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning-based models face reliability issues due to imbalanced data distributions in training datasets, where minority classes suffer from low predictive accuracy, and existing under-sampling techniques fail to consider the contribution of each data sample effectively.
Innovation Solution
A method that involves training a first-level ML model, calculating the contribution value of each data sample, and omitting redundant samples based on these values to create a balanced target dataset for training a second-level ML model, thereby improving dataset quality and model reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If existing under-sampling techniques are used to handle imbalanced classes, then the quantity of majority class samples is reduced, but the quality of training data is compromised because redundant samples are not effectively identified and removed
Solution Approach 1:
The system employs an automated self-service mechanism where the ML model itself evaluates and ranks data samples based on their contribution to training. The model autonomously identifies redundant samples and generates omission recommendations without human intervention, allowing the system to self-optimize the training dataset composition.
Solution Approach 2:
The patent replaces manual or rule-based mechanical sampling methods with an intelligent system that uses ML model evaluations and contribution calculations. Instead of fixed mechanical under-sampling rules, the system dynamically selects which samples to remove based on their calculated contribution values to training reliability.
2Adaptability or versatility
If data samples are randomly removed from majority class to balance dataset, then class distribution becomes more balanced, but predictive accuracy for minority classes deteriorates due to loss of valuable information
Solution Approach 1:
The system performs preliminary evaluation of each data sample's contribution to training before removal. By calculating contribution values in advance and ranking samples, the system ensures that only truly redundant samples are removed, preserving valuable information that would otherwise be lost in random sampling approaches.
Solution Approach 2:
The patent changes the parameter of sample selection from random to contribution-based ranking. By introducing a new parameter (contribution value to training) and using it to determine which samples to remove, the system transforms the sampling process from blind random removal to intelligent selective removal that maintains predictive accuracy.
3Quantity of substance
If all data samples from initial dataset are used for training, then comprehensive coverage is achieved, but training efficiency decreases and model reliability is compromised due to imbalanced class distribution
Solution Approach 1:
The system extracts and removes redundant data samples from the initial dataset based on contribution evaluations. By taking out only the necessary portion of majority class samples that do not contribute meaningfully to training, the system achieves balanced class distribution while maintaining comprehensive coverage of important data patterns.
Solution Approach 2:
Instead of using all available data samples (excessive action), the system applies partial action by selectively removing a portion of majority class samples. This partial removal is optimized to achieve the right balance between maintaining comprehensive coverage and improving training efficiency through better class distribution.
Data Source
AI summary
A system and method of training a machine-learning (ML) based model by at least one processor may include receiving an initial dataset, including a plurality of annotated data samples; based on the initial dataset, training at least one ML-based first-level model to perform a first-level task; based on training the at least one ML-based first-level model, calculating at least one characteristic, representing, for each data sample, a value of contribution into a training of a ML-based second-level model to perform a second-level task; omitting a subset of data samples from the initial dataset based on the at least one characteristic, to obtain a target dataset; and training ML-based second-level model, to perform the second-level task, based on the target dataset.