Deep Learning Ensemble Pruning for Bias-Reduced Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning classifiers suffer from biases such as representation bias, aggregation bias, and Simpson's paradox, leading to inconsistent and biased predictions, particularly in equipment maintenance and other data-driven systems.
Innovation Solution
A method is developed to create an ensemble of neural network models using intelligent sampling and dynamic pruning to reduce these biases, leveraging existing training algorithms without increasing training time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single deep learning model is used for predictions, then the training time is reduced, but the predictions suffer from biases such as representation bias, aggregation bias, and Simpson's paradox
Solution Approach 1:
The patent divides a single deep learning model into multiple separate models (ensemble), where each model is trained on different subsets of data or with different configurations. This segmentation allows each model to capture different patterns and reduces the impact of biases present in any single model, thereby improving prediction reliability while managing complexity through modular architecture.
Solution Approach 2:
The patent combines multiple individual model predictions through aggregation mechanisms (such as averaging, voting, or stacking) to produce a final prediction. This merging of multiple models' strengths compensates for individual model weaknesses and biases, improving overall prediction accuracy and reliability while the combination strategy manages the complexity of having multiple models.
2Reliability
If multiple models are trained to form an ensemble, then bias reduction is achieved, but the training time increases
Solution Approach 1:
The patent performs preliminary actions by pre-processing the training data into multiple subsets or pre-defining model configurations before actual training begins. This preliminary preparation enables more efficient parallel training of ensemble models and reduces the overall training time while still achieving bias reduction through the diverse model ensemble.
Solution Approach 2:
The patent implements partial training strategies where models are trained for a limited number of epochs or on subset of data rather than complete training. This partial action approach reduces training time significantly while still capturing sufficient patterns for each model, and the ensemble aggregation compensates for the incomplete training of individual models, achieving bias reduction with acceptable time investment.
Data Source
AI summary
Example implementations described herein are directed to systems and methods for generating a model ensemble to reduce bias, the method involving training a plurality of machine learning models from data, each of the plurality of machine learning models trained from a first subset of the data and validated from a second subset of the data, each of the first subset and the second subset being different for each of the plurality of machine learning models; determining accuracy of each of the plurality of machine learning models based on validation against the second subset of the data; pruning the plurality of machine learning models based on the accuracy to generate a subset of the plurality of machine learning models; and forming the model ensemble from the subset of the plurality of machine learning models.