Ensemble Weak Support Vector Machines for Classification Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models, such as support vector machines, often suffer from overfitting and underfitting due to insufficient training data, leading to low accuracy and precision in classification tasks.
Innovation Solution
The method involves generating multiple hyperparameter sets to create weak models, which are trained using bootstrap datasets and explanatory variable sets, and then aggregating these models to improve prediction accuracy by reducing overfitting and underfitting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single strong model is trained with limited data, then model complexity is reduced, but accuracy and precision deteriorate due to overfitting and underfitting
Solution Approach 1:
The patent segments the learning task into multiple weak models, each trained on different bootstrap samples and hyperparameter sets. Instead of one complex model, we create an ensemble of simpler models that each capture different aspects of the data, reducing individual model complexity while maintaining overall accuracy through aggregation.
Solution Approach 2:
The patent merges multiple weak models into a single ensemble predictor by aggregating their predictions. This combining approach allows the system to leverage the collective strength of multiple simple models, achieving high classification accuracy that would be unattainable by any single weak model alone.
2Measurement precision
If multiple models are trained to reduce overfitting, then classification accuracy improves, but computational time and resources increase
Solution Approach 1:
The patent creates multiple inexpensive weak models that are computationally cheap to train and evaluate. Each model is intentionally kept simple and 'disposable' in terms of computational resources, allowing rapid training on bootstrap samples without significant time investment, yet their collective prediction provides robust accuracy.
Solution Approach 2:
The patent performs preliminary actions by training multiple weak models on bootstrap samples during the ensemble construction phase. This preliminary training on resampled data allows the system to pre-compute model predictions that can be rapidly aggregated during inference, reducing real-time computational burden while maintaining accuracy.
3Reliability
If hyperparameters are optimized to prevent overfitting, then model generalization improves, but training complexity and computational overhead increase
Solution Approach 1:
The patent systematically varies hyperparameter values across different weak models in the ensemble. By changing hyperparameters such as regularization strength, kernel parameters, and learning rates across multiple models, the system captures diverse generalization behaviors without requiring complex optimization of a single model's hyperparameters.
Solution Approach 2:
The patent introduces dynamics into the model ensemble by using different hyperparameter configurations for each weak model. This dynamic approach allows the ensemble to adapt to different data patterns and noise levels across bootstrap samples, improving generalization through diversity in model behavior rather than through complex static optimization.
Data Source
AI summary
In an approach to improving accuracy through weak model aggregation, one or more computer processors generating a plurality of hyperparameter sets, wherein each hyperparameter set in the plurality of hyperparameter sets contains one or more hyperparameters varied to increase over-training in one or more models, wherein over-training includes overfitting or underfitting. The one or more computer processors create a plurality of weak models utilizing a created bootstrap dataset in a plurality of created bootstrap datasets, a corresponding extracted explanatory variable set, and a corresponding hyperparameter set in the generated plurality of hyperparameter sets, wherein each weak model in a created plurality of weak models shares at least the created bootstrap dataset, the extracted explanatory variable set, the generated hyperparameter set, a machine learning technique, or a model architecture. The one or more computer processors predict a classification for an unknown datapoint by aggregating the created plurality of weak models.