Neural Network Parameter Mutation for Ensemble Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training multiple classifiers for ensemble methods in machine learning is costly and time-consuming, limiting the efficiency of achieving better accuracy in classification tasks.
Innovation Solution
A method that involves training a machine-learning-based classifier, dumping its parameters, recording change rates, and creating new classifiers by calculating dumped parameters plus change rates times random numbers, thereby reducing the need for additional training without compromising accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple classifiers are trained independently from scratch to achieve better ensemble accuracy, then the accuracy of the ensemble prediction is improved, but the training cost and time consumption increase significantly
Solution Approach 1:
The patent creates multiple classifiers by copying the parameters from a single trained classifier and adding small random perturbations. Instead of training multiple classifiers independently, the system generates ensemble members by copying the base classifier's parameters and introducing slight variations through random noise addition, thereby achieving diverse predictions without repeating the expensive training process
Solution Approach 2:
The patent modifies classifier parameters by adding small random values to the dumped parameters of a trained classifier. This parameter perturbation technique creates variations in classifier behavior while maintaining the core learned patterns, enabling the generation of multiple independent-looking classifiers from a single training run
2Adaptability or versatility
If multiple classifiers are trained independently from scratch to obtain independent predictions, then the independence of local predictions is improved, but the training cost increases
Solution Approach 1:
The system copies parameters from a single trained classifier multiple times, each copy receiving different random perturbations. This creates multiple classifiers that are parameter-wise independent while sharing the same training foundation, achieving prediction independence without the cost of multiple training processes
Solution Approach 2:
The patent performs parameter dumping and random perturbation generation in advance, before the actual ensemble prediction is needed. By pre-computing the mutated parameters and storing them, the system eliminates the need for repeated training when creating new classifiers, reducing both time and computational cost
Data Source
AI summary
A computer-implemented method is provided for reducing training costs for an ensemble of machine-learning-based classifiers. The method includes training, by a processor, a given machine-learning-based classifier from among the ensemble to obtain a trained classifier. The method further includes dumping, by the processor, parameters used to train the trained classifier to obtain dumped parameters. The method also includes recording, by the processor, change rates of each of the dumped parameters. The method additionally includes creating, by the processor without training, a new classifier from at least one other machine-learning-based classifier in the ensemble by calculating the dumped parameters plus change rates times random numbers for each local prediction by the trained classifier.