Composite ML Model for Unbalanced Data Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional machine learning models face challenges when dealing with unbalanced datasets, leading to issues of overfitting or underfitting, and existing approaches to address these problems often result in inefficient processing times, high memory usage, and reduced accuracy.
Innovation Solution
A computer-implemented method generates a composite machine learning model by creating multiple training datasets with balanced data subsets from an unbalanced dataset, using machine learning algorithms to train separate models, and combining these models for classification and prediction tasks, thereby improving accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional machine learning models are applied to unbalanced datasets, then the models can process data, but they produce biased outputs with poor classification accuracy due to overfitting or underfitting
Solution Approach 1:
The patent segments the unbalanced training dataset into multiple balanced subsets by dividing data from different classes into separate groups. Each subset contains balanced representation of classes, which are then used to train separate models. This segmentation eliminates the bias caused by direct training on unbalanced data while maintaining the ability to handle all classes effectively.
Solution Approach 2:
The patent creates a composite model by combining multiple individual models trained on different balanced subsets. Each model contributes specialized knowledge about specific class combinations, and the composite model integrates these perspectives to achieve high accuracy on the original unbalanced test dataset without the biases present in conventional single-model approaches.
2Reliability
If model complexity is increased to address underfitting on unbalanced datasets, then training accuracy may improve, but processing time and computational resources increase significantly
Solution Approach 1:
Instead of increasing the complexity of a single model to handle unbalanced data, the patent segments the data into balanced subsets and trains multiple simpler models on each subset. This approach achieves high training accuracy through proper data representation rather than model complexity, significantly reducing processing time and computational resource requirements.
Solution Approach 2:
The patent applies partial action by training models only on balanced subsets rather than the entire unbalanced dataset. This selective training approach provides sufficient information for accurate classification without the computational burden of processing all data with complex models, optimizing the trade-off between accuracy and processing efficiency.
3Reliability
If data equalization and regularization techniques are applied to unbalanced datasets, then model performance may improve, but computational burden and processing time increase
Solution Approach 1:
The patent segments the data processing task into creating balanced subsets from the unbalanced data, which eliminates the need for computationally intensive data equalization and regularization techniques. By manually constructing balanced training subsets, the approach achieves improved model performance without the high computational burden associated with conventional equalization methods.
Data Source
AI summary
Disclosed is a computer-implemented method for classification and/or prediction of data samples on unbalanced datasets, the method comprising: receiving unbalanced training dataset; generating at least one first model and at least one second model using unbalanced training dataset, the generation of the at least one first model comprises generating first training dataset comprising first data subset and second data subset of equal amounts of data; the generation of the at least one second model comprises generating second training dataset comprising third data subset and fourth data subset of equal amounts of data, machine learning algorithm(s) are employed for learning from the first and second training datasets; generating composite model using the first model(s) and the second model(s); employing composite model for classification and/or prediction on unbalanced test dataset for generating output, the output includes at least one classified data and/or a prediction for a data sample.