Shared Feature Extractor for Ensemble Learning Cost Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Ensembling multiple models in machine learning increases memory and computational costs due to the need for training and inference for each model, which can be prohibitive.
Innovation Solution
A machine learning model is generated by coupling a single feature extractor to multiple predictors, with the model trained using the ensemble outputs of these predictors, reducing costs by leveraging ensemble learning while only using predictors during training and deploying the trained feature extractor for inference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple models are ensembled for machine learning, then prediction accuracy is improved, but memory and computational costs increase in proportion to the number of models
Solution Approach 1:
The patent merges multiple predictors into a single neural network model where multiple prediction heads share a common feature extractor backbone. This consolidation allows the system to maintain the accuracy benefits of ensemble learning while reducing memory usage and computational overhead by sharing weights and parameters across all predictors, thereby resolving the contradiction between improved accuracy and increased resource consumption
Solution Approach 2:
The shared feature extractor serves multiple functions by extracting features that are utilized by all predictors simultaneously. This multi-functional design enables a single component to support multiple prediction tasks, reducing the overall quantity of computational resources and memory required while maintaining the ability to perform accurate predictions across different targets or classes
Data Source
AI summary
According to one embodiment, an information processing apparatus includes a processor. The processor generates a machine learning model by coupling one feature extractor to each of a plurality of predictors, the feature extractor being configured to extract a feature amount of data. The processor trains the machine learning model for a specific task using a result of ensembling a plurality of outputs from the predictors.